From September 2006 to December 2008 the Government Accountability Office (GAO) reviewed several weapon programs and identified problems resulting in billions of dollars in cost overruns, long schedule delays and reduced weapon system availability. The major reason for these reliability problems was poor systems engineering practices related to requirements analysis, design and testing.

The GAO findings were summarized in the GAO-08-294 report published in February 2008, “Better Practices: Increased Focus on Requirements and Oversight Needed to Improve DOD’s Acquisition Environment and Weapon System Quality.” While the Department of Defense (DoD) planned to invest approximately $1.5 trillion toward its major weapon systems portfolio, it recognized the need to improve reliability issues for several of the weapons programs. According to the report, poor reliability led to significant problems for numerous weapons programs.

The report stated that the Marine Expeditionary Fighting Vehicle (EFV) faced such significant quality problems that DoD extended development by four years at an estimated cost of $750 million. A key reliability requirement that the EFV failed to meet was the 17-hour rate between mission failures. The EFV demonstrated a capability of 7.7 hours before it experienced a failure – well short of the time required in early vehicle testing.

In addition to problems with the EFV, the GAO also identified reliability problems with the Army’s advanced threat infrared countermeasure jam head (part of the Army’s missile warning system). The problems forced the Army to initiate a major redesign of the jam head in fiscal year (FY) 2006 and delay fielding the subsystem until FY 2010. Also, the Joint Air-to-Surface Standoff Missile program experienced a number of flight test failures that have underscored product reliability as a significant problem.

Although major defense contractors have adopted commercial quality standards in recent years, quality and reliability problems persist in DoD weapon systems. Current business practices from Six Sigma and other business initiatives may help practitioners working with DoD to improve quality and reliability. One of the tools, Design for Six Sigma (DFFS), allows developers to identify potential problems at the conceptual design phase of a project. A team of DoD contractors used DFSS to design a reliable ammunition rotational device to be used on a ground combat vehicle for the Army’s future combat systems.

Simulation Model

The rotational device stores the munitions for the ground combat vehicle. The rotational device is indexed to a specific location in order for the autoloader to load the munitions into the weapon. A functional requirement of the device was that it must allow for a munition to be loaded in 8 seconds under all operating condition. A derived requirement of 2.5 seconds to transfer time was placed on the rotational device. A design team was identified to develop an experiential design study to identify the critical parameters to meet this requirement.

Although there were a number of factors that could be tested in the time-to-transfer process, the team conducted a cause-and-effect analysis and selected the test factors given in Table 1:

The team selected these factors based on prior experience and an engineering knowledge of the design of the systems, and because of the interaction between them. From a variety of design types and analysis strategies the team decided on a full factorial design. The team selected the design because:

1. Conducting the deterministic model was inexpensive.
2. They wanted to eliminate all aliasing effects.
3. The design is orthogonal.
4. All main factors and interactions can be evaluated.

The first application of the rotational device system model described above was to perform a design of experiments (DOE) study of the effects of various systems and environmental factors on time-to-transfer munitions. This was done by simulating a servo model that rotates the rotational device to a reference position of 180° ± 2 mm in 2.5 seconds or less.

Simulation Results

The full factorial experimental design was conducted as specified by the team. The resulting Pareto diagram is shown in Figure 1.

Within the Pareto diagram, the items are displayed by frequency, in descending order. Most of the effects that contributed to transfer time were the number of munitions in the magazine – as the amount of munitions decreased the time to transfer decreased.

The discrete event simulation result shown below show the baseline and normality of time-to-transfer for the rotational device. As the results show, there were 578,064.197 defects per million opportunities (DPMO), which means that nearly 58 percent of the time the rotational device cycle time was greater than 2.5 seconds. The expected value analysis (EVA) result also showed a sigma level of -0.197, which was below the target level of 6 sigma (Figure 2).

Figure 2: Expected Value Analysis of Time-to-transfer for Rotational Device Process Inputs Process Outputs Normal Distribution Statistics Observed Defect Statistics Factor Distribution First Second Para. Para. Motor back EMF contstant Normal 0.391 0.013 Motor torque constant Normal 1.117 0.037 Motor resistance Normal 0.243 0.008 Ambient temperature Uniform -32 71 Munition loading Uniform 1 27 No.of Simulations 100,000 Mean 2.5273 StdDev 0.1387 USL 2.5 DPMO 578,064.197 Cpk -0.066 Sigma level -0.197 Sigma capability -0.197 Actual defects 55,797 95% Conf. inv lower 554,885.283 95% Conf. inv upper 561,051.354

The results from the parameter design did not reach the desired level of DPMO. The next step in the evaluation was to determine if the team could influence the cycle time requirement. The user determined that the team could decrease the cycle time of the rotational device. The revised time-to-transfer upper specification limit was increased to 2.8 seconds.

After performing the discrete event simulation with the new upper specification limit of 2.8 seconds the results showed a DPMO of 436.182 – nearly 99.7 percent of the time the rotational device cycle time was less than 2.8 seconds. The discrete event simulation result also reflected a sigma level of 3.3, which was below the target level of 6 sigma, but an improvement from the previous runs (Figure 3).

Figure 3: Expected Value Analysis with Revised Time-to-transfer for Rotational Device Process Inputs Process Outputs Normal Distribution Statistics Observed Defect Statistics Factor Distribution First Second Para. Para. Motor back EMF contstant Normal 0.429913 0.013 Motor torque constant Normal 1.2283466 0.037 Motor resistance Normal 0.219 0.008 Ambient temperature Uniform -32 71 Munition loading Uniform 1 27 No.of Simulations 100,000 Mean 2.4273 StdDev 0.112 USL 2.8 DPMO 436.182 Cpk 1.11 Sigma level 3.329 Sigma capability 3.329 Actual defects 0 95% Conf. inv lower 0.0 95% Conf. inv upper 36.888

By increasing the derived requirement of transfer time of the rotational device to 2.8 seconds it decreased the DPMO from more than 550,000 to less than 450. This new approach did not add any additional weight to the system or take additional space in the design.

Increasing Reliability and Robustness

This research showed how simulation was used to develop a robust design on the rotational device on a ground combat vehicle. A simulation model provided an inexpensive way to do this sensitivity analysis. It answered the question of how robust a design is with respect to the specification of the factors in a robust design experiment. The end result was a tool that allowed for a designer to develop a new system with a higher degree of reliability and robustness.