Many organizations are awash in data, generating enormous and plentiful reports with a variety of statistics. Others have little data to work with, often going by gut feel and experience when decisions are to be made. Most are somewhere in between. In the transactional world, many process measurement systems are manual, comprised of spreadsheets, Access databases, and, yes, even clipboards with tick sheets.

Let’s assume for the moment that you are working in an organization that actually has decent process-metric data to work with. The next question invariably is, “can we trust this data?” or, “are we even measuring the right things?”

To keep this entry brief, let’s assume that the right things are being measured. We may even have operational definitions, and a process by which process measurements are taken. (Naturally, in the absence of these things, I recommend starting with a CT Flowdown to identify the big Y’s, then process map to identify key process variables, create operational definitions for them, and then document and implement a new measurement system for our process.)

Some Master Black Belts insist that a full Gage R&R Study is required for all measurement systems, in order to validate the reliability of the data the project is based on. Is this always necessary for a Green Belt, or even possible?

In an MBB class I took last year, our instructor suggested that there are three options when it comes to validating our data: MSA, A&V, and H&P. MSA, is fairly self-explanatory. A&V refers to Audit and Verify: take a sample of your data and audit for accuracy with your process personnel. Does it pass the laugh test? If so, it’s probably reliable enough to move forward with. If not, you know this needs work before moving ahead. The last option, H&P is the least desirable: Hope and Pray, and is our default position if the MSA and A&V options are skipped. Try this one at your next tollgate meeting before the Champion – let us know how that turns out.

So when should we MSA, and when is an A&V sufficient? Like most things, it depends. In an effort to keep DMAIC project work lean, I tend to start with A&V before determining whether a full-blown MSA is needed. If a high degree of precision is needed in the measurement system, or measurement variation poses a large risk, absolutely run an MSA. MSA is also very useful where humans are making attribute or quality determinations, as in an inspection step or categorizing an input for further processing, or if each operator has “their own way” of doing things. Often, the Attribute MSA is the only way to illustrate the deficiencies in that measurement system. In the case of a destructive MSA situation, where repeatability is not possible, the study can still point out where reproducibility is a problem, as well as agreement with the standard.

So let me ask you, dear readers, how do you handle Measurement System Analysis, especially for your transactional projects? Please post your experiences, suggestions and/or horror stories in the comments section.

Special thanks to BMG Instructor George Rommal for introducing the A&V and H&P concepts.