If you want to make valid data based decisions, you need to have confidence your measurement system will provide reliable and consistent results. Reproducibility is one measure of your measurement system’s ability to provide trusted data.
Overview: What is reproducibility?
Measurement System Analysis (MSA), is a formal statistical study that determines whether your measurement system is capable of providing reliable data so that you can make the best possible data-driven decisions. The statistical study used for continuous data is called a Gage R&R study. The R’s stand for Repeatability and Reproducibility.
In reality, you never directly know whether your measurement system is providing accurate and precise measurement. When you measure an item, the value that the measurement system provides is a combination of the natural variation of the item plus the variation in the measurement system.
Repeatability is a measurement of the variation when one person repeatedly measures the same item, using the same measurement device and method. If the measurement device, whether it be a gage, scale or even a human is consistent with small variation, then the observed variation of the item can be attributable to the variation of the item rather than the measurement device.
Reproducibility is a measure of the variation when multiple people repeatedly measure the same item, using the same measurement device and method. Reproducibility would be the amount of observed variation attributable to between person variation. The graphic below shows what good and poor reproducibility would look like.
An industry example of reproducibility
Jane, a Six Sigma Master Black Belt (MBB) was reviewing the results of a MSA study to see if there were any issues with repeatability and reproducibility. Below are some of the graphs she was analyzing.
Jane noticed that, while the Total Gage R&R was acceptable (less than 10%), the reproducibility component was almost twice that of repeatability (2.40% vs 1.29%). This indicates the gage was consistent, but there was some variation between the three operators in how they measured the study objects.
Reproducibility may be seen if there are significant differences in the operator patterns generated by each operator measuring the same samples. Jane concluded the patterns were sufficiently similar to claim good reproducibility.
Finally, good reproducibility is when the average of all measurements between operators is similar. The line connecting the three boxes below is essentially horizontal indicating the average of the operator measurements shows good reproducibility.
Frequently Asked Questions (FAQ) about reproducibility
Why is reproducibility important?
It describes the variation in your measurement system attributable to person to person variation. If you have multiple people taking multiple measurements of the same object, you would want the variation between them to be very small, meaning a high reproducibility for your measurement system.
How is reproducibility different from repeatability?
Repeatability is the same person repeatedly measuring the same objects with the same device and method. Reproducibility is multiple people repeatedly measuring the same objects with the same device and method.
If my measurement reproducibility is poor what can I do about it?
The first question to ask is why are you using multiple people for your measurements? Can you use less or even one person? You can also try to provide better operational definitions of how to measure an object.
