Process Improvement at Raytheon

Background

Raytheon is an international technology based company; the Software Systems Laboratory division employs around 400 engineers. Since 1988 they have been trying to improve their software processes. This has proven to be successful, as shown by the following results:

• a $7.70 return on every dollar invested;
• a two-fold increase in productivity;
• an evolution from level 1 (Initial) to level 2 (Repeatable) to level 3 (Defined) process maturity, as measured by the Capability Maturity Model.

Approach

Raytheon has developed an approach for process improvement based on a three-phase cycle of stabilization, control and change, which applies the principles of Deming and Juran.

1. In the process stabilization phase the emphasis is on distilling the elements of the process actually being used and progressively institutionalizing these across all projects.
2. In the process control phase the emphasis shifts to instrumenting projects to gather significant data and analysing the data to understand how to control the process.
3. In the process change phase, the emphasis is on determining how to adjust the process as a result of measurement and analysis and how to distribute the new methods among practitioners.

Improvement is continuous; thus completion of the third phase signals a beginning of the first.

Calculating savings

During the first five years of the SPI initiative funds have been used to support a staff of only one or two full-time personnel, plus many people working part-time - on average the equivalent of about 15 people per year. At one time, about 100 people were involved in one process-improvement task or another. From the very beginning of the initiative measurement of results was started. Using Crosby's approach a differentiation was made between costs of doing it right the first time and the cost of rework. Categories of the costs associated with any process were defined as:

• Performance; the costs associated with doing it right the first time.
• Appraisal; the costs associated with testing the product to determine if it is faulty.
• Rework; the costs associated with fixing defects in the code or design.
• Prevention; the costs incurred in attempting to prevent the fault from getting into the product.

The sum of appraisal, rework and prevention costs is what Crosby calls "the cost of quality". The total project costs is simply the cost of quality plus performance costs.

The rework costs at the time the initiative started averaged about 41% of the total project cost. After a five year period they had shrunk to about one fourth of its original value (from 41% to 11%). At the same time the appraisal costs have increased by about 5%, as a percentage of the total project costs. Although the increased rigor with which design and code inspections were conducted accounted for some of this increase, most of it was due to a 30-percent decrease in the total project costs, which pushed up appraisal cost proportionally. The Return-On-Investment (ROI) has been calculated for six major projects, employing about 58 percent of the total available labour, and receiving the benefit of the $1 million investment. The total savings over the six projects in reduced rework was $4.48 million, thus the return on investment was approximately 7.7 to 1 (a $4.48 million return on a $0.58 million investment).

Productivity improvements

Also productivity data was obtained in the analysis, in terms of equivalent delivered source instructions per man month. Source code size was calculated with a tool that counts lines in most common languages (Ada, C, Fortran and Pascal). Modified and reused lines of code are weighted according to the relative effort of modifying or reusing it. During the period of the initiative the productivity increased by a factor of about 2.3 in about four and a half years. Typically projects are all of the same type - real-time embedded - and within a reasonable size range 70.000 to 300.000 lines of code.

Less tangible results

In addition to the earlier mentioned project characteristics, a number of equally important but less tangible results have been achieved:

• software engineers are spending fewer late nights and weekends on the job, despite increasing demands of new business;
• communication has improved, higher levels of interpersonal contact and job satisfaction;

Finally a number of contracts have been awarded to Raytheon because they can show their customers they work on a higher (CMM) maturity level.

References

Dion, R. (1993), Process Improvement and the Corporate Balance Sheet, in: IEEE Software, July 1993

Other CMM case studies can be found in:

Humprey, W.S. (1991), SPI and Hughes Aircraft, in: IEEE Software, 1991

Wolwend, H, et al (1994), Schlumberger's Software Improvement Program, in: IEEE Software, 1994