Six Sigma project metrics are typically taught in detail during Black Belt and Green Belt training classes. But the inconsistent use of these metrics after training can lead to lost opportunities and rework after a project ends. Using project metrics consistently brings enormous clarity to the impact and benefit of a Six Sigma project and helps keep project teams focused on enhancing business capabilities.
Defining Project Metrics
One common mistake in the use of project metrics is to phrase the project goal to realize a certain saving in a given area, rather than a process improvement. This confuses the primary metric with financial metrics. Another inconsistent use of metrics is found among the secondary metrics, which are sometimes misunderstood as representing the project Champion’s extended wish list. For example, an early version of my first Black Belt project charter stated the primary metric as improving delivery speed. Delivery quality and hours worked were considered secondary metrics with improvement targets of their own, rather than side effects of the primary metric.
Table 1 summarizes the standard Six Sigma project metrics, their meaning, their relation to the primary metric and their use in improvement projects.
| Table 1: Six Sigma Project Metrics | |||
| Project Metric | Definition | Correlation to Primary Metric | Use in Improvement Project |
| Primary Metric | Defines the project goal: “improve (primary metric) from (baseline) to (target) by (date)” | N/A | Measure baseline and improvement level when the project ends |
| Secondary Metric | Captures, validates and tracks welcome side effects of the project | Strong (high relative predictive power, or R2). By default, there is a cause-and-effect relationship between the primary and secondary metrics. | Measure baseline and impacts of project after improvement. Monitor during and after project if linked to financial metric. |
| Consequential Metric | Captures, validates and tracks non-welcome side effects of the project | Strong (high R2). In the current design of the process, there is a cause-and-effect relationship between the primary and consequential metrics. | Collect data before, during and after the project to prove no collateral damage was caused by the project. |
| Financial Metric | Links progress in the primary and secondary metrics to financial advantage. Most often this metric is tailor-made for the specific project | Expected strong (high R2). | Evaluate at project milestones (i.e., Define, Measure, Analyze and Control phases) and typically one year after the project ends. |
| Business Metric | Measures how an organization achieves one of its major goals. | Typically weak (low R2). | Monitoring is optional and mainly needed for communication purposes. |
Selecting the Primary Metric
Selecting the appropriate primary metric for an improvement task is the result of a sound process to identify project scope. A well-scoped project focuses on only one primary metric while considering the impact of the other types of metrics. One way to identify the right primary metric for the project is to use a balanced scorecard approach.
A sound project statement is phrased using the primary metric. For example, “The project goal is to improve (primary metric) from (current performance) to (desired future performance) by (desired date of completion).”
Many projects begin with an objective such as, “Our document archiving does not work properly and must be improved.” It is important to impose the necessary discipline on project Champions and process owners to transform such vaguely phrased problem statements into proper project statements.
Connection to the Business Metrics
Why is an improvement in the primary metric good for business? Why, for example, are faster deliveries better? The relation between the primary metric and the underlying business metric answers this question. Faster deliveries may increase customer satisfaction, which is one of the top-level goals of an organization. Thus, there is a known cause-and-effect relationship between the primary and business metrics.
However, a business metric such as customer satisfaction is influenced by many factors other than faster deliveries. So, most often the strength of the correlation between the primary and business metric is weak and the regression coefficient R2 is typically small and difficult to measure. When the average lead time for delivery is improved from five days to three days, many deliveries may still last longer than desired. This is a case of achieving a primary metric goal while leaving the business need unsatisfied.
For business metrics such as customer satisfaction, indirect statistical methods, including Taguchi’s, can provide a useful roadmap. Notice that a project can impact several business metrics correlated to the primary metric or to any of the secondary metrics. For practical reasons a project typically quantifies the relationship to only one business metric. This helps the project team understand and focus on a single organizational goal.
Benefit of Using a Secondary Metric
Secondary metrics quantify welcome side effects caused by improvements in the primary metric and are typically linked to the primary metric by default. Reducing the lead time to issue an offer, for example also reduces the number of offers in progress at any given moment.
Often secondary metrics measure progress in areas not directly affected by the project. Validating and tracking these metrics can build broad-based support for the project. To validate a secondary metric, quantify its cause-and-effect relationship to the primary metric and ensure its relationship to a financial or business metric is transparent.
Impact of Consequential Metrics
This situation can be illustrated with an example of a project that aims to issue faster offers. In the current design of the process, quickly issuing an offer degrades the quality of the offer. A non-diligent preparation of offers produces defects, which degrade the offer-to-project conversion rate (because the offer is overpriced) or the project margin (because the offer is underpriced). In his book It’s Not Luck (North River Press, 1994) Eliyahu Goldratt introduced the “evaporating cloud,” a way of dealing with seemingly mutually exclusive goals – such as speed and quality – in a project Writing down the conflict and the links between the statements allows practitioners to identify ways to break one or more of the links. Using the evaporating cloud method may push the project team to question whether it is true that offers prepared quickly cannot be offers prepared diligently. Asking questions such as this can lead the team to invent new design options for the process rather than simply improving the existing process.
It is a good idea to establish a measurement system for consequential metrics during the Measure phase of a project. Tracking these metrics after project finalization will provide ongoing information on whether the previously existing link between the primary and consequential metrics is broken in a sustainable way.
Importance of Financial Metrics
Evaluating the benefits of a project includes establishing a financial metric for improvements realized by the project. Saving money is not always the primary interest of Lean Six Sigma projects. When the primary metric is tied to a business metric, it is often enough to know that the project enhanced the company’s business capability. Even so, it makes sense for organizations to have a monitoring system in place for the results of their continuous improvement activities across all areas. The financial metric is not used for project accounting but rather to translate the project impact into a generalized business benefit.
The financial metric of the project should be reassessed at all project tollgates. Many companies also continue to validate this metric one year after project finalization.
Six Final Tips
From these considerations arise the following recommendations for Six Sigma project managers, Master Black Belts, process owners and project Champions:
- Derive one primary metric for the project team to work on for each improvement project.
- Use the primary metric to formulate the project goal.
- Brainstorm for possible secondary and consequential metrics. Select and validate a set of few essential metrics.
- Link at least the primary and consequential metrics, and eventually also the secondary metrics, to business metrics. Validate and quantify the cause-and effect relationships.
- Construct a financial metric based on the primary and secondary metrics.
- Make sure all metrics fulfill the requirements to facilitate the creation of a data collection plan.
