In any financial service process that is being studied for the first time, it is common for Six Sigma teams to spend one-third to one-half of their project time on data collection alone. It simply takes a lot of time to figure out what data is needed, to develop a reliable data collection process and to analyze the results. Here are three tips that can help teams get a fast start on data collection.
1. Monitor Some Baseline Metrics
Every improvement project will have its own unique data needs, but there are a number of metrics that are useful for determining overall process health, such as:
Work in process (WIP)/Things in process: The amount of work that has entered the process but has not been completed.
Average completion rate: The average number of work items that are completed in a given time period (a day, an hour, etc.).
Demand variation: The amount of fluctuation in the demand for the output of the process. The amount of work that arrives at a given activity is measured in terms of units/day and as an average completion rate per unit. Variation can be used in queuing theory to estimate the resulting delays that this variation causes.
First-pass yield: The percentage of things in process that make it all the way through the process the first time without needing to be fixed or re-handled in some way. First-pass yield is a good overall indicator of how well the process is functioning. It also reflects both Lean and Six Sigma goals: in order to have a high first-pass yield, the process must operate smoothly (i.e., with good process flow) and with few errors.
Approvals or handoffs: Two characteristics almost always seen in slow processes are 1) a lot of approvals before work can be completed or 2) a lot of handoffs back-and-forth between people or groups. In contrast, Lean processes operating at high levels of quality are characterized by many fewer approvals and handoffs. While having low numbers of approvals or handoffs does not guarantee having a Lean process, this is relatively easy data to collect and will almost certainly drop as the process improves.
Defects/Sigma capability: The Sigma level in Six Sigma is, of course, the rate of defects that occur per defect opportunity. The key is to come up with definitions that: 1) everyone in the team will interpret the same way and 2) are consistent with other definitions used in the organization. For example, when filling out a new customer account form, is every keystroke counted as an opportunity for someone to make a mistake? Or is the whole form one “opportunity”? Do typos count the same as omissions? It is best to focus on things that are important to customers. There are a lot of ways that a form, a report or a service can be technically “defective” in some way without it mattering to internal or external customers. For example, perhaps different employees do the work in a slightly different sequence. If “sequence” affects quality as perceived by the customer, then doing the steps in the wrong order is a defect that should be tracked. If sequence does not affect the customer, then there are probably bigger fish to fry elsewhere.
Cycle or lead time: How long it takes for any work item to make it through the process from beginning to end.
Setup, downtime: Any delays or productivity losses that occur when people switch tasks.
Measure these things early in a project, then again once improvements have been made to determine the impact. If the process being measured has a lot of steps and/or a lot of throughput (volume of work), consider measuring on a sample basis at first (e.g., randomly sample key steps).
2. Observe the Process
In the words of Yogi Berra, “You can observe a lot by watching.” There simply is no substitute for impartial observation as a way to confirm what really happens in a process. Impartial observations are critical in identifying waste and inefficiencies built into how work is currently done.
In an office environment, it is hard to observe the work itself, since it can take the form of emails, reports, phone calls or inputs to screens – some work products may exist only in a virtual sense. As a result, process observation in service environments means watching people and what they do. And there is the rub. Not many people like someone sitting at their shoulder, watching their every move.
For that reason, process observation in financial services often works best with trained observers, especially if they are seen as neutral parties (i.e., they come from a different work area, they are a trained Black Belt who has not worked in the area before, etc.). Also, office staff needs to be involved in setting the goals for the observation (“What will be learned from this?”) and in deciding when the observation will happen, which staff will volunteer to be observed, and so on.
The example below is a form that Lockheed Martin found invaluable in the early stages of improvement projects. It was used for verifying (or refuting) everyone’s ideas about what they think is happening, and for helping them zero-in on areas that need attention.
3. Collect Data by Participating in the Process
What better way to evaluate a particular service than by acting as a customer of that service? This can be done most easily by having a selected group of employees physically walk the process pretending to be an item of work – an incoming customer call, for example. What happens to the record of the call? Who works on it next? What happens at that work station? Where does it go after that?
Another approach is shared by Roger Hirt, a Black Belt with the City of Fort Wayne, Ind., who recalls one project where a team wanted to improve the quality of response to citizen calls. Instead of doing an after-the-fact survey of callers, they used secret shoppers, people who interact with the process just as real customers would. First, they provided the secret shoppers with standard scripts relating to different types of inquiry and complaint calls. They had these people call the city department at different times (so they would talk to different staff), then looked at how the staff had handled the calls. They discovered a lot of inconsistency in how staff recorded and categorized information, with the result that citizens weren’t always provided with correct answers or responses. This information allowed the department to develop training for everyone who received calls. A second secret shopper trial showed dramatically improved results.
Collecting data this way is a sensitive issue. On one hand, the idea is to be assured that the secret shoppers are getting service similar to what real customers would experience. But on the other hand, employees may be disturbed if the data collection comes as a complete surprise. Project Black Belts and improvement teams will have to make a judgment call about how much to tell people ahead of time.
Conclusion
These tips demonstrate that collecting data in service environments involves much more than simply getting out a stopwatch or counting errors on a form. A project team will make more progress if it plans to observe the process in question, pay attention to where and how the data is collected, and be creative in finding ways to evaluate customer experience with a particular service.
