“You don’t know what you don’t know.” That phrase has been taught in Black Belt training for many years, and it continues to be relevant today. When combined with a related phrase, “What you don’t know can hurt you,” these words illustrate a common challenge: Sometimes, the events that hurt us are unknown and unexpected. It may seem like an obvious statement, yet so often the very processes that can mitigate the risk of the unknown are the first ones to be discarded.
The Thrill of Discovery
Part of what can make Six Sigma (or any continuous improvement effort) so exciting is the gratification of seeing real-world improvements as a result of one’s efforts. The carrot is the thrill of discovery that tempts practitioners with the notion that new knowledge will provide the power (knowledge really is power) to change our environments.
But things just are not that simple. True, Six Sigma does involve maximizing knowledge. However, how much one knows is not the only thing that influences success. The ability to manage the risk inherent in what is unknown is another important factor.
Insurance Tools
Multiple DMAIC tools can help manage the risk of what is unknown – they are self-correcting or insurance tools. Because they do not directly appeal to the quest for knowledge and the promise of power, however, they tend to be the first tools to be mistakenly discarded in efforts to “streamline” the problem-solving process. The rest of this article examines some of these tools and the impact of their neglect on Six Sigma projects throughout each phase.
Define
Charter Development: Most projects have some form of a charter, so the complete neglect of the charter is an issue as much as a lack of diligence. Vague problem statements, poorly considered team selection and careless estimation of financial or strategic impact are the most likely elements to be neglected in the charter. When these tools are not addressed properly, the entire focus, direction and legitimacy of the project are at risk. Lack of attention to effective team selection can severely impede effective team facilitation and decisions.
Secondary Metric: Typically this tool is part of the charter, but it is noted separately simply because of the frequency with which it seems to be omitted from projects. Inasmuch as this metric ensures that positive change in the primary metric does not negatively affect another part of the business, the secondary metric is nothing more than an insurance policy. Consequently, this metric is one of the first tools to be excluded. Fortunately, effective pilots and communication among team members and stakeholders to review findings and proposed solutions can compensate for the omission of this metric.
Stakeholder Analysis: This Define phase tool addresses risks related to change management and acquisition of resources (particularly human). It is employed to understand the political landscape with regard to the project and provide guidance for ensuring that resistance is minimized. Depending on the scope and nature of a project, the exclusion of this tool will fail to address potentially pernicious organizational resistance.
Voice of the Customer: Too often customer requirements are assumed to be adequately known; the illusion of knowledge here is both frequent and severe. Lack of diligence in the set of tools that typically fall into the VOC genre can produce misguided (at best) and dangerous solutions. In the DFSS arena, lack of effective VOC gathering and analysis is especially treacherous because it will nearly always cause requirements failures, released design failures and customer dissatisfaction.
Team Facilitation: When teams are either poorly facilitated or meetings are not held at all, the quality of decisions throughout the project will be diminished. This will in turn significantly increase the risk that important root causes will be missed, solutions will not be well conceived and people affected by the changes will not accept them.
Measure
Measurement System Analysis: Second only to the secondary metric, this particular tool tends to be one of the most neglected and underused of all the key tools in the DMAIC/DMADV project cycle. This is unfortunate because poor decisions and related project failures are caused by bad data more than any other failure mode. MSA can take many forms, ranging from verification of operational definitions to logical inquiries and analyses of the data collection process to sample audits for accuracy to gage R&R. Some form of MSA, minimally as a basic investigation of the data collection systems, is a standard for well-executed projects primarily because of the value it brings to ensuring the integrity of decisions.
This is, aside from team facilitation, the single most important tool for reducing the risk of the unknown, but because it does not, by itself, fix processes or products (it only sharpens the tool that allows you to fix them), MSA is underappreciated and easily ignored in efforts to streamline the DMAIC/DMADV process.
FMEA: For practitioners who have used the FMEA [failure mode and effects analysis] to discover, validate or prioritize potential root causes and detection/control issues on a process that has never been documented with the tool before, it is the single most painful and difficult team tool (aside from maybe quality function deployment) that is employed. You get what you pay for, however, and when facilitated properly it brings tremendous value to process improvement.
The relentless attention to detail required by the FMEA makes it unparalleled in its ability to train people to think critically about their processes. This enables the effectiveness of the Six Sigma method by first revealing process opportunities and then by forcing team discipline and attention to detail, so it is both a discovery tool and an insurance tool. It increases what is known about the process at the same time that it ensures that what is unknown is minimized. Occasionally, Six Sigma practitioners are seduced by the convenience of other team process discovery tools like fishbones, Y-to-x trees and cause-and-effect matrices, convincing themselves that the FMEA is unnecessary because it cannot provide anything beyond the other team process-discovery tools. This is a dangerous assumption because it fails to account for the value of the FMEA in driving critical thinking and ensuring that all bases are covered.
Team Facilitation: This topic is reintroduced here in the Measure phase because this phase is influenced so thoroughly by team-oriented process analysis tools. The effective facilitation of teams is instrumental in creating an effective list of critical process inputs.
Analyze
Analysis Plan: This is a basic guidance document that is typically produced at the beginning of the Analyze phase and updated throughout it. It is intended to provide direction by way of a documented trail of results and conclusions from the various studies. It helps the team (including the Champion) to remain conscious of where they are and where they are going in their analysis journey. This document is a tool that can drive decision-making in light of all previous knowledge gained and avoid decisions made in a vacuum.
Power and Sample Size: Most people recognize that one can show that just about any difference or relationship is significant if the sample size is large enough. This is all about managing a-risk. Two problems typically arise here: the neglect of the power (b-risk) of the test and consideration of the fact that the samples may not be very random, which throw the results of the test out the window. Of course there are many opportunities errors to be made in the various analyses of this phase, but effective team facilitation and logical or intuitive consideration of the results tend to overcome the risks.
Residual Analysis: As the single most effective test to ensure that assumptions of normality or homogeneity of variance hold, this test is often omitted, which can be a significant source of poor decisions. Again, the logical or intuitive assessment of the results and the involvement of the project team to review the results can overcome this omission, but given the ease and the effectiveness of the test, there does not seem to be any coherent justification for its omission.
Team Facilitation: Aside from the disciplined adherence statistical assumptions noted above, this is the most effective tool available in this phase to ensure integrity of the analytical conclusions. All too often the Belts will insulate themselves from the team and immerse themselves in analysis alone.
Improve
Mistake-Proofing: When training is relied upon to solve problems, the issue will not be eliminated. Significant reduction of sources of waste nearly always involves mistake-proofing elements. Enough said.
Pilot/Prototype (Simulation): Pilots or prototypical solutions are often the best tools available to help discover what is unknown and assist in the identification of potential risks in a solution. Sometimes pilots or prototypes are not feasible and in a sense they might be considered “waste” because they force redundancy, but the information they yield can be invaluable and often not obtainable through other means. Simulation is a leaner form of a pilot and should be used where pilots are too expensive.
Team Facilitation: As in the other phases, team facilitation continues to ensure that good decisions are made through conscientious consideration and review by multiple individuals.
Control
Control Plan: The entire Control phase in Six Sigma is basically a self-checking and self-correcting system. For some people it lacks the thrill of other phases because there is not the same kind of learning and discovery, but the real value in the Control phase is the application of solutions. This is where the control plan is so critical: a system to ensure that solutions are implemented correctly, maintained properly and optimized through additional learning. The control plan is just another inspection and Plan/Do/Check/Act cycle, and it is critical to the vitality of any Six Sigma effort.
Project Post-Mortem: As the control plan is a critical self-correcting tool for individual projects, the project post-mortem is an important technique to ensure that deployments are continuously improved. Unfortunately, this particular tool is frequently neglected due to the perceived lack of immediate value derived or due to the fact that the information is not used by the deployment management team.
Conclusion
There is no single technique that exists to ensure decision-making integrity. In reality, there are several tools and disciplines that exist as checks-and-balances, some quantitative in nature, others structural. The risk comes into play when these self-correcting elements are marginalized. Too often Champions and Belts approach their problems with an illusion of knowledge and a bias for what the solution should or ought to be. Or equally as dangerous, when they believe the process can be just as robust in a “reduced” form.
Problem-solving techniques that focus on acquiring new knowledge without managing the risk of what remains unknown may seem more efficient, but they are also much riskier. A practitioner may know a lot, but that knowledge is less relevant if he or she does not deal with what is unknown. One cannot be satisfied only with what one knows. Continually question the extent of what is unknown. More often than not, it is the unknown that causes project failures.
