Defining “Pilot”
The meaning of acronyms and words change over time – for example, ARM for many was an acronym for allergy relief medicine; banks use the acronym for adjustable rate mortgages. The same holds true for the meaning of pilot – the light that kept stoves burning, the title of an airplane captain or “pilot” as used within Six Sigma implementation efforts. Pilot can be classified as a:
- Noun: an experimental or a preliminary trial or test – process improvement teams will pilot the solution within the improve phase of the project.
- Verb: to lead, steer, guide or conduct, as through unknown places, intricate solutions – The Green Belt will pilot the solutions prior to full implementation.
- Adjective: serving as an experimental or trial undertaking prior to full-scale operation or use – a pilot project.
From a Six Sigma viewpoint a pilot is defined as testing the functional and sigma capabilities of the new process, where critical functions defined in the current generation of the multi-generational plan are operational but on a limited scale.
Four primary advantages and/or objectives of utilizing a pilot are:
- Limit capital and other resource expenditures (managing risk)
- Assess true performance of design and/or solutions in a controlled but “live” environment
- Identify additional improvements
- Identify implementation tips and traps
If implementation difficulties are encountered during the pilot, they will likely recur during full-scale implementation. Team leaders can identify these difficulties in a live but controlled environment, and take action to fix them prior to the actual implementation.
Why Pilot?
There are numerous reasons to pilot:
- Confirm expected results and relationships
- Understand expected variation in the process and its possible impact to the customer
- Improve a solution
- Improve the implementation
- Lower the risk of failure
- Improve the ability to better predict monetary savings from a proposed solution
- Increase opportunities for feedback
- Increase buy-in
- Quickly deliver a version of a solution to a particular segment
- Validate the measurement system
Consider two examples of Green Belt projects. The first project had a productivity goal of 85 percent. The Green Belt conducted a pilot and found that productivity would reach 96 percent. Although this clearly surpassed the goal of the project, the Green Belt used information from the pilot to modify the solutions. After full-scale implementation, the project was performing at 100 percent.
The second project’s pilot was conducted during the improve phase of the project. It was a full-scale pilot. The results not only verified the solutions, but also accelerated the completion of the project through the control phase because there was nothing left to implement. The pilot was the implementation.
When to Pilot
In general, there is a way to pilot all or some part of every solution or new design that an organization wants to implement. It is almost always worth the extra effort to pilot. Consider doing a pilot when:
- The scope of the design is large,
- The new product/service could cause far-reaching, unintended consequences,
- Implementing the design and/or solutions will be a costly process, and
- The design and/or solutions would be difficult to reverse.
If conducting a DFSS (Design for Six Sigma) project, as opposed to a DMAIC (Define, Measure, Analyze, Improve, Control) project, piloting is almost always a must.
Pilot Key Considerations
Determining how long a pilot should run and how many samples are necessary are important considerations to be certain enough evidence is collected. Project leaders also need to be confident that the process is stable with regard to its performance over time. If the process is not stable the process capability (the ability to meet your customer requirements) cannot be assessed.
Evaluate the measurement system first since it is imperative that the data collected is accurate and precise. Then assess the stability of the process prior to evaluating the capability. Lastly, review the scorecards.
Verify the Implementation Plan
After the pilot, verify the implementation plan. Obtain assistance from the team and the process owners. These resources can help answer questions such as: was your schedule met? Were the instructions clear? Were they followed? What additional information did people need? Were there unexpected challenges encountered?
Pilot Verification and Validation
Once the implementation plan has been verified, verify and validate the pilot itself. Analyze gaps between predicted and test performance and pilot results. Analyze gaps between pilot results and actual requirements. And also analyze the actual pilot project plan; how did things come together? Did the design of the scorecards predict capability of the overall process and elements? For the gaps identified, perform a root cause analysis to determine “why.”
Piloting Summary: Tips
The design team should be there as much as possible during the pilot process. What they learn and observe will be worth the time they invest. Collect data on process and external factors that may be influential. If possible, make sure that the full range of inputs and process conditions, including expected variation in input and process variables, are tested in the pilot. Define successful verification before piloting; agree on a list of critical parameters and targets. Most important – communicate. Not just pre-pilot but also post-pilot. This keeps everyone well informed to either address additional opportunities for improvement or help celebrate success, which will promote additional buy-in for full-scale implementation.
