Following the implementation of Capability Maturity Model Integration (CMMI) at Level 5 in an organization, the normal practice is to measure the metrics for each process area and report it to top management. The overall performance of the organization in terms of a specific process area is not measured in most cases.

A single performance measure or sigma value may be required when an organization wants to share its success stories with existing or new clients.

The sigma level of an organization in terms of key CMMI process areas can be measured by identifying suitable metrics that relate to that particular process area. Organizations that have not deployed Six Sigma also can follow this technique to measure their performance level in terms of project planning process area.

Project Planning Process Area

The project planning process area has three specific goals and five generic goals. The specific goals are:

1. Establish estimates.
2. Develop a project plan.
3. Obtain commitment to the plan.

The generic goals are:

1. Achieve specific goals.
2. Institutionalize a managed process.
3. Institutionalize a defined process.
4. Institutionalize a quantitatively managed process.
5. Institutionalize an optimizing process.

Assuming the generic goals are the first and foremost criteria which need to be met in order to achieve CMMI Level 2, the need is to focus on the specific goals and specific practices to arrive at the correct measure and metrics so that project planning performance or sigma level can be measured in a correct and optimized manner.

Specific practices for the specific goals for project planning are:

1.1. Establish the scope of the project.
1.2. Establish the estimates of work product and task attributes.
1.3. Define project life cycle.
1.4. Determine estimates of effort and cost.

2.1. Establish the budget and schedule.
2.2. Identify project risks.
2.3. Plan for data management.
2.4. Plan for project resources.
2.5. Plan for needed knowledge and skills.
2.6. Plan for stakeholder involvement.
2.7. Establish the project plan.

3.1. Review plans that affect the project.
3.2. Reconcile work and resource levels.
3.3. Obtain plan commitment.

Identified from the Specific Practices

Here are potential features/measures/metrics for project planning (CMMI Level 5) identified from the specific practices listed above:

• Work breakdown structure based on the product architecture and tasks
• Identified risks
• Risk prioritization based on probabilities and impacts
• Risk mitigation plan and its quantification (failure mode and effects analysis)
• Work products that will be externally acquired
• Work products that will be reused
• Estimation criteria
• Life cycle and its applicability
• Project issues and constraints
• Resource planning and its effectiveness
• Planned effort versus actual effort (effort variance)
• Planning schedule versus actual schedule (schedule variance)
• Estimated cost and actual cost
• Knowledge and skill requirements
• Stakeholders involvement plan
• Data management plan
• Project plan effectiveness – audit non-conformance reports
• Number of revisions of project plan
• Configuration management for project planning
• Corrective action taken for metrics deviation and issues
• Improvement opportunities identified
• Tailoring of project planning process

While project planning performance or sigma level can be calculated based on one or two important metrics, it is more practical if a tradeoff analysis can be conducted by identifying additional metrics and collecting more data. As data verification of authenticity is a potential problem in a large or medium-sized organization, long-term data collection and proper data screening and measurement system analysis will help in arriving at a correct sigma value.

The project planning sigma level can be calculated based on the following metrics, which typically are either measured on a regular basis in any organization or collected from the project teams or tools.

Commonly Measured Metrics

The measures/metrics in the table below are useful if a large or medium-sized organization wants to calculate the sigma value for project planning. Customer specifications, organizational specification or industry averages can be considered to arrive at the specification limit.

Calculation of Sigma Value

Here (and in the figure below) is the process for calculating the sigma value for project planning:

Step 1: Identify development projects from different customer accounts. Segmentation is done in terms of technology and domain.

Step 2: Identify metrics from the specific practices of the project planning process area. As all the metrics are not measured and for some metrics, data availability and data correctness are a problem, only commonly measured metrics are considered.

Step 3: While some metrics are measured, upper and lower specification limits are not identified due to various reasons. Organizational performance in terms of those metrics is measured through a control chart process that utilizes plus/minus 3 sigma as the control limits. For those metrics, prior organizational data is analyzed and compared to the industry average to arrive at the specification limit.

Step 4: Collect the last eight to 12 months of relevant metrics data to visualize the organization level performance.

Step 5: Verify data or conduct measurement system analysis (manually) to check the correctness of the data.

Step 6: Screen data if data appears very erratic.

Step 7: Identify defects for each metric in terms of the specification limits. Units and opportunities are also identified for each metric.

Step 8: Add all the defects to arrive at the total defect. Add all the opportunities to arrive at the total opportunities.

Step 9: Calculate defects per opportunity and yield in terms of total defect and total opportunities for project planning.

Step 10: Calculate sigma value for project planning process area for the organization from defects per opportunity.

Improvement Opportunity Identification

After measuring the baseline sigma level, improvement opportunities are identified to improve the overall project planning performance that will be aligned to the business objectives.

The typical improvement opportunities for project planning resulting from this methodology are:

1. Improvement in estimation effectiveness.
2. Reduction in schedule variance.
3. Effectiveness in risk management.
4. Improvement in customer satisfaction index in terms of project planning parameters.
5. Effectiveness in competency management.
6. Effectiveness in induction process.
7. Effectiveness in technology consumption.
8. Reduction in customer complaints.