Key Points

• Test plans are vital for any testing process.
• Plans should be detailed, blueprinting every part of the testing process.
• You’ll want to make sure they openly communicate every intention of your testing.

Test plans, also called test protocols, are formal documents that typically outline requirements, activities, resources, documentation, and schedules to be completed. Some form of test plan should be developed before any test.

The key reasons for developing test plans are:

• Preparation: To ensure that all reasonable aspects of running a test have been considered.
• Communication and Training: To train those who need to assist with the test.
• Effectiveness: To provide a mechanism for outlining test needs, limitations (listing assumptions), and justification for purposes of setting expectations, acquiring resources, investigating unexpected results, and assuring normalcy and effectiveness.
• Legal and Regulatory Prudence: To enable replication and protection of discoveries made, to mitigate potential litigation costs from the use of those discoveries, and to help provide evidence to show regulatory bodies of efficacy.

The Importance of a Plan

Before embarking on any testing, experimentation, and so forth, it is important to consider a plan. This gives you a blueprint of what you’ll expect throughout the entire process. Further, it acts as a safeguard, especially if you’re in an industry where compliance and regulations need to be considered.

Test Plan Content

Titles and order of sections within a test plan vary significantly from company to company, given differences in test types, strategy, scope, and industry.

Examples of tests that should have plans or protocols written for them include medical, non-medical, field, lab, or production studies, design of experiments (DOE), problem-solving, reliability or software regression tests, and design assurance, evaluation, or validation tests.

Beginning of the Plan (typically background-related information)

Information you’ll want to include at the beginning of the plan is header information, title, date, author, file code, project number, and product/device/model description. You might also consider sub-components of that product being tested, to whom the report is written, test number and revision, and references to other related documents.

References might include industry standards, brainstorm documents (fishbone diagrams, FMEA/FTA/FMECA studies, related V&V protocols and reports, field history or rejection reports, company procedures, related website links, etc.).

Cover pages and a table of contents should be considered for large plans. Most plans contain an explanation of what the test is about (i.e. summary, introduction, abstract, background, purpose, history, etc.). Some plans also include test type and strategy information, definitions, important terms or keywords, approval signatures, return on investment, and/or other justification information.

Middle of the Plan (typically scope-related information)

What you’re looking to include in the middle of the plan is a data collection or sampling plan (including sample size, confidence acceptance levels, and sampling techniques). You’ll also need to consider test conditions or setup instructions, a test procedure specifying exact measurements to be taken, test monitoring requirements, and resources (equipment and personnel) to be used including responsibilities.

Most plans contain some form of flow chart, process, or value stream maps. They also contain assumptions that need to be made and why (including statistical distribution related as applicable), a work breakdown structure or schedule, and/or how the test factors (response Ys and control Xs) were selected.

Some plans include special customer or site-related requests or considerations, measurement system capability information, and how test-related problems are to be reported and corrected (i.e. failure reporting and corrective action system). Also include unresolved issue reporting.

End of the Plan (typically analysis-related information)

The end of your plan should have statistical techniques to be used, the hypothesis to be tested, the power of the test matrix and/or why it was chosen (for DOE), and a definition for test success (pass/fail criteria as possible).

Most plans include how the data is to be analyzed, contingencies for how to handle different types of preliminary or insufficient results, and how the conclusions are to be reported or summarized (i.e. significance with confidence, pass/fail, etc.). Some plans also include references and/or bibliography, appendices that might be needed, and copyright or proprietary information messages as applicable.

Generic Test Plan Advice

• Make Them Context Rich: Context-rich test plans contain seemingly unimportant details that often do help reduce confusion, document test details that might be needed, and/or help explain any unexplained results.
• Use Hypertext: It’s often helpful to add links to other parts of the same document (i.e. “jump to conclusion,” “see details,” “see references”).
• Add Key Assumptions: Listing key assumptions in the plan can be important for setting reader expectations, influencing approval, assuring management that the test is well thought out, explaining unexpected results, and providing direction for future tests. Example assumptions are as follows:
  • “Results are assumed to contain an even distribution of errors.”
  • “Temperature change stabilization times selected are assumed to be sufficient.”
• Convert Test Plan into a Test Report as Possible: Test plans can often be converted into test reports. Some sections only change in tense. Other sections can be reused exactly as they were written.
• Take Extra Data-Related Precautions: Ensure sampling is representative of the population (random and unbiased), personally observe data collection, and make sure the measurement system is capable.
• Develop Test Libraries: Standard tests and/or test results from prior tests (especially reliability tests) should be easily accessed and re-used as applicable and possible.

Other Useful Tools and Concepts

Looking for additional frameworks for your organization? You might consider learning how a Fishbone diagram works. This handy problem-solving tool is a great way of drilling down to the root cause of a problem in your production line.

Additionally, you might also want to take a closer look at the Harada Method. This method encourages employee self-reliance, which is key to developing a stronger workforce. Our guide on the matter gives some examples and how to implement it in your organization.

Conclusion: Test Plans a Sensible Step

It is easy to see how test plans are an important and sensible part of experimenting. They save time and money, assist in getting the best results, and can facilitate speedy test report writing.