An ami example

An application of ami in the computer manufacturing industry

This case study comes from the software development branch of a large European computer manufacturer. The company had some experience of collecting metrics but did not have a method to undertake a large-scale measurement programme. They selected the ami method since they perceived that it would increase the visibility of their software development process and would help them to measure and improve their product.

They started by applying the ami method to three of their projects that were using COBOL: a new development of about 44 man-months, and two maintenance projects of 120 and 150 man-months.

Problem description

The company's target was to produce a product that met its customers' requirements while delivering it within the deadlines and budget constraints. From this they targeted cost prediction, project scheduling, quality management and quality measurement as key areas.

Approach used

The company used their customer requirements to help them focus on the key areas instead of any SPI model, such as the CMM. However, they were not sufficiently thorough in validating their goals at this stage, which lead to some problems later in the programme.

The basic metric set in the ami handbook was used to help identify the metrics to be applied. They also used their earlier experience of collecting metrics to help identify cost-effective metrics. When they produced the metric plan they found that it was very large (120 pages). The structure was modified to shorten it and make it easier to update and evolve with their programme.

A program was written to help them collect some of the metrics. The rest were collected using forms. A spreadsheet was used to analyse the data and help produce graphical presentations of the data and metrics. The selection for a presentation style for the results needed some thought but eventually a tabular presentation was found to work best.

Goals and metrics used

The main goal was:

To improve the productivity while maintaining quality

In order to make this a more manageable the goal was broken down into sub-goals:

Evaluate productivity
Monitor quality
Detect impediments to higher productivity

The following tables give the metrics for sub goal 2 and the questions used to derive the metrics. Each table consists of a sub goal with a list of associated questions and the respective metrics. Subjective metrics are indicated in italic.

Sub Goal 2: Monitor Quality

Q1	How many residual errors are still in the system on delivery?
	Metrics: Number of errors detected during use Seriousness of each error detected Intermodule test coverage
Q2	What is the quality of the documentation delivered?
	Metrics: Number of documents delivered Number of documents reviewed Number of documents conforming to standard Rating of document quality Evaluation of the final quality assurance report
Q3	What is the quality of the documentation for post delivery support?
	Metrics: Number of documents used for support Number of documents reviewed Number of documents conforming to the company standard Rating of document quality Evaluation of the final quality assurance report
Q4	Are there any structural deficiencies in the software?
	Metrics: Number of calls Number of input/output calls Number of decisions per unit Evaluation of the final quality assurance report

Benefits

The company perceived that it benefited directly by getting increased visibility of its development process, improved cost estimates and better insight into software management. An indirect benefit was that the measurement programme contributed to them obtaining the ISO 9001 certificate.

Costs

The first project to which the company applied ami required about 6-7% of the whole project effort. This includes the installation of the metrics tool and familiarization with MS EXCEL. In the subsequent projects this decreased to 2%, and they expect to bring this to below 1% in future projects.

Conclusions

The company concluded that the ami method made them think in a top-down way and proceed from strategic objectives to more concrete aims. In the early stages they were very enthusiastic and used many goals. The ami method showed that the goals they had used to derive metrics were at too high a level and needed more careful analysis. This included identifying more knowledge goals. The ami method has given them the experience and confidence to continue to extend and expand their metrics.

References

Pulford, K., A. Kunntzmann-Combelles and S. Shirlaw (1996), The ami handbook, A quantitative approach to software management, Addison Wesley, ISBN 0-201-87746-5