Key Points

• Design of Experiments are tools used to see how certain things will impact a process.
• There are two main approaches to DOE, full factorial and fractional factorial.
• Before conducting the main DOE, you’ll want to screen to eliminate noise from your data sets.

Design of experiments (DOE) can be defined as a set of statistical tools that deal with the planning, executing, analyzing, and interpretation of controlled tests to determine which factors will impact and drive the outcomes of your process. 

This article will explore two of the common approaches to DOE as well as the benefits of using DOE and offer some best practices for a successful experiment. 

Overview: What Is Design of Experiments

Two of the most common approaches to DOE are a full factorial DOE and a fractional factorial DOE. Let’s start with a discussion of what a full factorial DOE is all about.

The purpose of the full factorial DOE is to determine in what settings of your process inputs will you optimize the values of your process outcomes. As an example, if your output is the fill level of a bottle of carbonated drink, and your primary process variables are machine speed, fill speed and carbonation level, then what combination of those factors will give you the desired consistent fill level of the bottle?

With three variables, machine speed, fill speed and carbonation level, how many different unique combinations would you have to test to explore all the possibilities? Which combination of machine speed, fill speed, and carbonation level will give you the most consistent fill? The experimentation using all possible factor combinations is called a full factorial design. These combinations are called Runs.  

We can calculate the total number of runs using the formula # Runs=2^k, where k is the number of variables and 2 is the number of levels, such as (High/Low) or (100 ml per minute/200 ml per minute). 

But, what if you aren’t able to run the entire set of combinations of a full factorial? What if you have monetary or time constraints or too many variables? This is when you might choose to run a fractional factorial, also referred to as a screening DOE, which uses only a fraction of the total runs. That fraction can be one-half, one-quarter, one-eighth, and so forth depending on the number of factors or variables. 

While there is a formula to calculate the number of runs, suffice it to say you can just calculate your full factorial runs and divide by the fraction that you and your Black Belt or Master Black Belt determine is best for your experiment.

Benefits of DOE

Doing a designed experiment as opposed to using a trial-and-error approach has several benefits. 

Identify the Effects of Your Main Factors

A main effect is the impact of a specific variable on your output. In other words, how much does machine speed alone impact your output? Or fill speed?

Identify Interactions

Interactions occur if the impact of one factor on your response is dependent upon the setting of another factor. For example, if you ran at a fill speed of 100 ml per minute, what machine speed should you run at to optimize your fill level? Likewise, what machine speed should you run at if your fill speed was 200 ml per minute? 

A full factorial design provides information about all the possible interactions. Fractional factorial designs will provide limited interaction information because you did not test all the possible combinations. 

Determine Optimal Settings for Your Variables

After analyzing all of your main effects and interactions, you will be able to determine what your settings should be for your factors or variables. 

Why Is DOE Important to Understand?

When discussing the proper settings for your process variables, people often rely on what they have always done, on what Old Joe taught them years ago, or even where they feel the best setting should be. DOE provides a more scientific approach. 

Distinguish Between Significant and Insignificant Factors

Your process variables have different impacts on your output. Some are statistically important, and some are just noise. You need to understand which is which.

Existence of Interactions

Unfortunately, most process outcomes are a function of interactions rather than pure main effects. You will need to understand the implications of that when operating your processes. 

Statistical Significance

DOE statistical outputs will indicate whether your main effects and interactions are statistically significant or not. You will need to understand that so you focus on those variables that have a real impact on your process.

Why It Matters

When you sit down to run any sort of statistical analysis, you need an approach that works. Sure, you could run trial-and-error testing, but that paints an incomplete picture while also being rather time-consuming. Instead, having a proven method of testing a variety of factors in a DOE paints a far more comprehensive picture, allowing you to focus on what matters in your processes.

An Industry Example of Design of Experiments

A unique application of DOE in marketing is called conjoint analysis. A web-based company wanted to design its website to increase traffic and online sales. Doing a traditional DOE was not practical, so leadership decided to use conjoint analysis to help them design the optimal web page.

The marketing and IT team members identified the following variables that seemed to impact their users’ online experience: 

• loading speed of the site
• font of the text
• color scheme
• primary graphic motion
• primary graphic size 
• menu orientation

They enlisted the company’s Master Black Belt to help them experiment using a two-level approach.

In a conjoint analysis DOE, you would create mockups of the various combinations of variables. A sample of customers were selected and shown the different mockups. After viewing them, the customer then ranked the different mockups from most preferred to least preferred.

The ranking provided the numerical value of that combination. To keep matters simple, they went with a quarter-fraction design or 16 different mockups. Otherwise, you’re asking customers to try and differentiate their preferences and rank way too many options.

Once they gathered all the data and analyzed it, they concluded that menu orientation and loading speed were the most significant factors. This allowed them to do what they wanted with font, primary graphics, and color schemes since they were not significant.

Best Practices with Design of Experiments

Experiments take planning and proper execution, otherwise, the results may be meaningless. Here are a few hints for making sure you properly run your DOE. 

Carefully Identify Your Variables

Use existing data and data analysis to try and identify the most logical factors for your experiment. Regression analysis is often a good source of selecting potentially significant factors. 

Prevent Contamination of Your Experiment

During your experiment, you will have your experimental factors as well as other environmental factors around you that you aren’t interested in testing. You will need to control those to reduce the noise and contamination that might occur (which would reduce the value of your DOE).

Use Screening Experiments to Reduce Time and Cost

Unless you’ve done some prior screening of your potential factors, you might want to start your DOE with a screening or fractional factorial design. This will provide information as to potentially significant factors without consuming your whole budget. Once you’ve identified the best potential factors, you can do a full factorial with the reduced number of factors.

Other Useful Tools and Concepts

If you need more help with your DOE, you might want to look into our tips and tricks on how to build a practical one. It covers a basic full factorial DOE using eight-run arrays. It might not be the most comprehensive means of constructing one, but it serves as a good foundation for further experimentation.

Further, learning the ins and outs of full factorial design is a must for anyone looking to check the impact of independent variables on their outputs. We touched on full factorial throughout today’s article, but that doesn’t paint the whole picture.

Conclusion

A design of experiments (DOE) is a set of statistical tools for planning, executing, analyzing, and interpreting experimental tests to determine the impact of your process factors on the outcomes of your process. 

The technique allows you to simultaneously control and manipulate multiple input factors to determine their effect on a desired output or response. By simultaneously testing multiple inputs, your DOE can identify significant interactions you might miss if you were only testing one factor at a time. 

You can either use full factorial designs with all possible factor combinations, or fractional factorial designs using smaller subsets of the combinations.