Despite encouraging results with various quality improvement
approaches, the IT industry is still far from achieving zero
defect software. Testing will remain an important activity
within software development and maintenance, often taking more
than 30 - 40% of the total budget. Both the increasing
importance of software in the society and the costs that are
involved in testing, confirm the need for structuring the
testing process. This paper provides an outline description of
TMap, the Test Management approach for structured testing (both
white-box and black-box) of software products. It provides
answers to the what, when, how, where and who questions of
testing. To structure the organization and execute the test
processes TMap is based on four cornerstones:
usable techniques for the various testing activities (T).
The four cornerstones for structured testing
In recent years TMap has evolved towards the standard for
software testing in The Netherlands. It is being used by more
than two hundred Dutch organisations. Most Dutch banks,
insurance companies, pension funds and government departments
use TMap partly or as a whole. More and more SME's have adopted
TMap and new market segments have been penetrated such as
consumer electronics, telecommunications and logistics. The
TMap book (in Dutch) has proven to be a best seller,
international interest and awareness has resulted in a plan for
the release of an English version.
Pitfalls
Studying the state-of-the-art and the state-of-the-practice
highlights a number of pitfalls regarding structured testing.
These pitfalls will be discussed and also the way in which TMap
tries to deal with them:
- Time
Time is quite often the major problem for testing. However, by
starting testing activities early by establishing a master test
plan during the requirements phase, a well-founded testing
strategy and effort estimation and a thorough preparation of
test cases can contribute to solving this problem.
- Testing strategy
A major problem when testing is the prioritization of the
various tests and the co-ordination between the various types
of test. Discussion with all parties involves according to a
structured process will establish clear priorities between the
various parts of the system and quality characteristics. Again
a master test plan covering the co-ordination and interfaces
between the various types of testing will be very useful
- Organization
Often there is no real commitment towards testing and there is
a lack of knowledge on the subject. In recent years a number of
good testing books have become available and training courses
are run regularly. A half-day management awareness training
can be especially beneficial.
- Test object
During testing the test object often looks like a moving
target, requirements still change and various software versions
are "available". Take advantage of good development practices
and software process models, such as the CMM; a configuration
management should be established supported by adequate tools.
Testing has to make use of the controlled development
environment.
- Testing environment
Modern day testing requires an infrastructure, e.g. a testing
environment that is more and more difficult to realize as a
"as-if-production" environment. Starting on time can partly
solve this problem since there is then still more time
available to realize the needed infrastructure. However,
sometimes a "as-if-production" environment is just not possible
for time and/or cost reasons. It is the duty of the test
manager to make the risks involved clear to general management,
e.g. the project sponsor.
- Testing techniques
With new development tools and rapidly changing hardware
traditional testing techniques no longer seem appropriate to
use. In these circumstances adaptations have to made to the
generic testing process to cope with RAD, Client/Server, the
year 2000, etc.
- Testware
For maintenance release the staff involved in testing during
development are often no longer available. The establishment of
testware during a (planned) completion phase at the end of the
testing process can considerably reduce this problem. The
re-use of testware is one of the most important benefits of
structured testing.
References
Pol, Teunissen and Veenendaal, Testing according to
TMap (in Dutch), ISBN 90-72194-33-0, Tutein Nolthenius
(NL), 1995
Trienekens and Veenendaal, Software Quality from a
Business Perspective, ISBN 90-267-2631-7, Kluwer
Bedrijfsinformatie (NL), 1997
Veenendaal and Pol, A Test Management approach for
Structured Testing, in: Achieving Software Product
Quality, ISBN 90-72194-527, Tutein Nolthenius (NL), 1997
