Since the start of the information technology age, software quality has been an ambiguous term, meaning different things to different people. It has been defined internally from the viewpoint of software developers; and it has been defined externally from the viewpoint of end users of the software. But either way, early in the history of software development it was obvious that software could not be developed in an ad hoc manner but needed to be subject to rules/methodologies to deliver a product or service that had attributes like robustness, correctness, adaptability, reusability and maintainability.

During the last couple of years numerous software development methodologies have been introduced to guide development teams in achieving these quality goals. Some methodologies involve a disciplined and detailed process with strong emphasis on planning. Examples of such methods include capability maturity model (CMM), software process improvement and capability determination model (SPICE), team software process (TSP), personal software process (PSP), etc. Then recently some Agile methods have been advocated as a new paradigm for high-speed, quick-to-market software development. Examples include extreme programming (XP), SCRUM, etc. The basic critique of the agile software development evangelists is that conventional software development processes are too rigid to achieve the end-results let alone the aimed quality factors for contemporary projects.

But one thing is sure: no matter which process/methodology is applied, the bottom line is that there is an impact of software processes on the software quality factors.

Software/IT Success-and Failure Rates

In what most consider a mature software/IT industry, the success/failure rate is not as good a one might expect. In fact, surveys by professional bodies like the Standish Group, Robbins-Gioia, KPMG, OASIG, etc. reveal a dismal picture. Some results from the 1995 Chaos report by the Standish Group found out the following:

• Some 31 percent software projects were cancelled before completed.
• About 52 percent projects cost almost double of original budget estimates.
• Only 9 percent of projects were on-time and within budget.
• On average, more than 50 percent of the effort of producing software went into testing.
• More than 50 percent of the costs associated with software development were incurred after delivery.

Even though the numbers tended to improve with the later reports, the top reasons found for project failures are still the same:

• Lack of user input
• Incomplete requirements and specifications
• Changing requirements
• Lack of executive support

All this leads to the conclusion that the business value of IT is still only vaguely perceived. And with the current watch on technology investments, there is even greater need for obtaining a good return on investment (ROI) on money spent on technology. A disciplined approach to software generation is quintessential if this goal is to be accomplished.

Quality Has to Reflect on the Bottom Line

When one looks at an industry structure, it becomes apparent that the focus of the players depends on the relative competitive positioning. In a monopoly or an oligopoly (few key players), the companies typically look at their cost and add a mark-up (to get the sought-after market volume) to get to a certain selling price. The equation looks like:

Cost + Profit = Selling Price

But as the industry structure moves towards greater competition with many players providing almost a commodity product/service, the perspective of the equation changes:

Selling Price – Cost = Profit

It is a different paradigm when the players have similarly priced products. The only way to gain competitive advantage is to focus on minimizing the costs that reduce the bottom line. Minimizing costs is exactly where the IT and application development industry is. The current critical nature of software and application development in a product or service offering has elevated the importance of the software process in ROI considerations. Especially for any new IT initiative, there is a lot of effort to insure a satisfactory return on investment.

It can be a challenge to summarize the cost and benefit factors and values that contribute to estimating ROI in any project. This is where Six Sigma offers a set of tools for tracking and enabling success. The Six Sigma value proposition is directly tied to positive financial results, and hence the methodology often has the buy-in of top management. This focus on financial results ensures that executive buy-in remains through the execution and rollout phase of a project, the lack of which is a major reason of failure of IT projects. It also ensures business discipline in project selection and project tracking, with a focus on maximizing the benefits delivered to the bottom line. The Six Sigma philosophy is that satisfied customers come back for more and encourage their business associates, family and friends to do the same. It recognizes that the ultimate quality award is improved bottom-line profitability.

Using a Holistic View of All Costs

As suggested in Juran’s Quality Handbook, analysis of many application development projects has consistently revealed two facts:

1. Almost half of the development resources are used for testing.
2. Almost 70 percent of total resources over the life of the software are used in maintenance.

Hence, cost savings can be realized only with methodologies that yield high quality initially and manage the change control process. This becomes even more relevant with the recent emphasis on a fundamental decision-support tool called “total cost of ownership.” This tool requires a holistic view of all costs across the organization that will be incurred throughout the life cycle of the project, including people, processes and technologies.

The typical cost drivers of a Six Sigma initiative for software process improvement that are essential inputs into the ROI estimating process, include:

• Training costs, labor hours and travel costs
• Project costs, activity costs and administration
• Costs of customizing processes
• Response conditioning costs
• Appraisal costs (those resulting from examining activities for execution errors)

Typical benefits of such an effort include:

• Better quality with fewer defects
• Less rework and lower maintenance costs
• Faster cycle times – quicker time-to-market for new products
• Higher productivity, leading to lower development and design costs
• Higher business value (typically more product value to customers)
• Higher customer satisfaction leading to higher revenues

So What Is Six Sigma for Software?

Six Sigma practitioners follow these tenets as a business philosophy:

• If something cannot be measured, we really do not know much about it.
• If we don’t know much about it, we cannot control it.
• If we cannot control it, we are at the mercy of chance.

And so Six Sigma, in general, measures deviations from an ideal. The measurable thing may be the process used to make a toothbrush, whose material specifications and costs are measured and compared to ideal material specifications and ideal cost. Or it may be the process of a stock brokerage, where quality of service and cost of service are measured against ideals of quality and service. When it is applied to IT, Six Sigma measures attributes like network speed, network reliability and relevant line of business processes.

Thus, measurements are made to gain an understanding of processes to see what works and what does not; to evaluate, predict and determine status with respect to plans; and to improve (rational use of quantitative information to identify problems and strategies to remove them).

Six Sigma started in the manufacturing industry with emphasis on management of efficient processes, efficient management of people, dedication to measurement systems, etc. But it became apparent that business success was more than the absence of negatives (defects, delays, cost overruns). Six Sigma then began to encompass positives like customer loyalty and delighters in new products. From operational excellence, Six Sigma has moved towards customer intimacy and product leadership value disciplines.

There are essentially two approaches to ensure software quality:

• Assurance of the process by which the product is developed.
• Evaluation of the quality of the end product.

There is always need to study both the quality of the software product and the life-cycle cost incurred in the development and maintenance of the products. The quality of software has been studied mainly from defect analysis and software maintenance perspectives. The effect of the process used in a software project on the outcome of the project in terms of cost to the software developer and quality of the product is key. And this is where Six Sigma for software comes in. It hold that the process of generating the product is more important than the product itself.

Application Development Not Like Manufactuing

But an IT person will immediately point out, application development is not like manufacturing since process variation can never be eliminated because:

• No two modules are alike and so performance includes an intrinsic degree of variability.
• There is great variation in human cognitive processes – differences in skills from one developer to another.
• Quality assurance processes work differently for manufactured goods than they work for software. This is because software products need testing only once before being called “acceptable,” and it is the same “acceptable” program that is distributed to customers. Thus, unlike a traditional factory where there are many different independently produced units that are each tested, a software factory produces one program.

Yet the software application development processes can be fully characterized by three simple measurements:

• Time: The time required to perform a task.
• Size: The size of the work product produced.
• Defects: The number and type of defects, and fixing time.

Six Sigma for software is not a software development process definition – instead it is a general methodology for improving processes and products. It provides a holistic quality objective in product development, combining a programmer’s desire for bug-free piece of software and a user’s desire for an easy-to-use application that fulfills a business need. Although a few elements of the Six Sigma toolkit are invoked within other methodologies like the Software Engineering Institute’s PSP/TSP framework (e.g., regression analysis for development of estimating models), there are many other tools available in the Six Sigma for generating good quality software products.

For instance, voice of the customer (VOC) and quality function deployment (QFD) are useful for developing detailed and prioritized customer requirements (both stated and unstated). More than just sending out survey forms, these tools involve creating meaningful customer feedback. VOC is captured in a variety of ways – direct interviews, surveys, focus groups, customer specifications, observations, etc. This understanding of the customer needs is then summarized in a matrix or QFD. These tools ensure that the following necessary steps are taken care of:

• Identifying customers in a given business scenario.
• Recognizing the benefits of clearly defining customers’ requirements.
• Creating a critical-to-quality needs tree in a given business scenario.
• Determining the appropriate Kano model category on which to focus customer improvement efforts.

Then there are numerous charting/calculation techniques that can be used to scrutinize cost, schedule and quality (project-level and personal-level) data as a project proceeds. For technical development, there are quantitative methods for risk analysis and concept/design selection.

Once management has driven the business strategy down to projects, one can work on them with the Six Sigma DMAIC approach – define, measure, analyze, improve, control. This starts with the assumption that all work is a process. And the goal is to improve the process in general. But Six Sigma applications in software also reach beyond the business goal of improving current processes/products and extend to design of new processes/products. Six Sigma incorporates the Design for Six Sigma (DFSS) approach to improving the feature/function/cost trade-off in definition and design of the software product. Tools such as KJ analysis, QFD, conjoint analysis, design of experiments (DOE) and many others have high leverage applications in the world of new product development in software.

Six Sigma and Other Methodologies

There are many models that have been used for disciplined application development and the key to understand is that the disparate quality models need to be evaluated in the context of business objectives. Project characteristics that determine the relevance of a particular methodology can be broadly based on following dimensions:

Project Size: This is not only the measure of lines of code or megabytes of data, but primarily the complexity of the system, the nature of its main components, interactions and – importantly – the size and composition of the development team.

Application Domain: This includes not only the domain of environmental management but also the characteristics in more technical terms – real-time, distributed (client/server architecture with both LAN and WAN elements), decision support objective, etc.

Criticality: Projects of high criticality like the ones doing forecasting for weather conditions, or decision-support systems in important domains like emergency healthcare need rigorous methodologies to ensure the quality.

Innovation: If a project is a research project, involving a number of new and partly not yet proven technologies, like 3D dynamic simulation, a fair amount of process customization can be done to eliminate need to follow rigid procedures.

Figure 2 provides a matrix that suggests a way of selecting a model for a certain project.

Heuristically, it has been found in the application development industry that Six Sigma for software begins to be cost-effective once the organization has reached CMM Levels of 4 or 5. As an organization matures to those levels, it begins to truly leverage established measurement practices, and that is when accomplishment of true Six Sigma performance becomes a relevant goal. And at that point, Six Sigma and approaches such as CMM/CMMI, PSP and TSP become complementary and mutually supportive. From a corporate strategy viewpoint, Six Sigma is an enabler to launch CMM, CMMI, PSP, etc. with similar tools, methods and concepts of continuous improvement. And yet it differs from CMM/CMMI in that:

• Six Sigma speaks the language of business, addressing costs, profitability and ROI.
• Six Sigma projects are drawn from a portfolio of problems that are identified through business strategy while CMM linkage to strategy is weak.
• Six Sigma teams consist of highly trained employees from many departments of the company, not just quality assurance. (called Boundary-less collaboration).
• Six Sigma is typically applied to selected projects, while CMM/I/PSP/TSP are intended for all projects across the organization.
• Six Sigma makes it impossible, like with practices such as CMM, to fall into the trap of laying a veneer of process over the same old activities.
• Six Sigma may be applied to the improvement of many other business practices other than the primary goals of CMMI/PSP/TSP (software product cost, cycle time and delivered quality). Such practices include improved customer service after delivery of the software, improved customer satisfaction and value realization from the software product feature set delivered. Six Sigma applies to the software process, the software product and to balancing the voice of the customer and the voice of the business (the stated and unstated needs of the businesses/shareholders) to maximize overall business value resulting from processes.

The most fundamental tenet of Six Sigma is that the business must be “managed manage by fact.” This view is consistent with that of TSP/PSP, but it has not yet been established that PSP/TSP is the best alternative in every context, only that it is better than some alternatives. Six Sigma can help organizations find the solutions that are truly optimal for each unique circumstance.

Conclusion: More Like Other Parts of Business

As information technology becomes more involved as a business value driver in different industries, there will be improvements in IT’s own business processes. This will necessitate IT to become part of strategic processes and use Six Sigma concepts like business initiative evaluation, prioritization and approval, IT resource supply-and-demand balancing process, etc.