 |
Evaluation Services |
|
Evaluation Services |
Service addresses
certification bodies, test laboratories, customers, users, (suppliers)
Service description
As far as software product evaluation and certification are concerned, one of the main problems is that in many occasions views of quality can be different according to the specific application domain.
While the same general framework and the evaluation process can be applied, there are some product related aspects that can greatly differ, such as:
Quality profiles consist of the list of specific quality characteristics considered relevant for the software product, a sort of ranking of relevance of the different quality attributes proposed by a model.
It is evident that the definition of a unique quality profile, which is the same for every kind of application, is not meaningful.
Many software product evaluation methodologies now provide procedures to define, (e.g. through a questionnaire) the "right" expected quality profile for each kind of software product: an initial analysis of the client requirements for evaluation is usually carried out in order to identify the purpose of the service provided and the expected quality profile.
Once the definition of the expected quality profile has been completed, other procedures can also supply the "validation" of such profile in terms of applicability and suitability, as well as the identification of the adequate metrics/techniques/tools to be applied when performing the evaluation.
These procedures are intended to guarantee the objectiveness, the repeatability, the reproducibility and the impartiality of the methodology.
Examples of how an expected quality profile is defined can be found in:
signifies that this is a downloadable Word 6.0/95 document
|
The IBISCO initiative for the evaluation and certification of bank software product quality |
|
Software Product Quality Evaluation and Certification: the Qseal Consortium Methodology |
Service addresses
certification bodies, test laboratories, customers, users
Service description
Product quality models: a framework
Product quality must be considered from two sides: the customer needs and the internal technical characteristics (Fig.1). In order to cope with the complexity of the problem, all the quality models available in the technical literature (both past and present) decompose the external quality (the one perceived by the user) into attributes and then into sub-attributes (or characteristics and sub-characteristics, or features and sub-features, depending on the authors), and the technical choices are classified into technical characteristics, and then into sub-characteristics.
Fig.1 - Constructive definitions
The model is structured into three different parts:
Fig.2 - Definition, mapping, techniques
The currently accepted definition of the external quality is described by the ISO/IEC 9126: 1991 (Fig.3), which indicates the external attributes only, simply suggesting (in the annex) a possible decomposition of them into sub-attributes:
Fig.3 - ISO/IEC 9126: 1991
The International Standard ISO/IEC DIS 14598-5
The international standard ISO/IEC DIS 14598-5 provides guidance for the practical implementation of software product evaluation when several parties need to understand, accept and trust evaluation results.
In particular, it may be used to apply the concepts described in ISO/IEC 9126.
It defines and describes the activities needed to analyse evaluation requirements, to specify, design and perform evaluation actions and to conclude the evaluation of any kind of software product.
The standard defines the main characteristics of the evaluation process (repeatability, reproducibility, impartiality, objectivity) and describes the specific steps involved:
Examples of how evaluation can be carried out can be found in:
signifies that this is a downloadable Word 6.0/95 document
 |
ISO/IEC DIS 14598-5 - Information Technology - Evaluation of software product - Part 5: Process for evaluators |
|
The IBISCO initiative for the evaluation and certification of bank software products quality |
|
Software Product Quality Evaluation and Certification: the Qseal Consortium Methodology |
Service addresses
customers, users, suppliers, certification bodies, (test laboratories)
Service description
Certification is a procedure by which a third party gives written assurance that a product, process or service conforms to a specified set of characteristics.
Currently most software certification is related to the quality of the production processes rather than to the conformance of the products to their initial specifications.
Software product certification is typically based on the direct measurement of the product characteristics and performance against some defined set of requirements and functional specification.
The set of quality attributes stated by ISO/IEC 9126 (functionality, reliability, usability, efficiency, maintainability and portability) is the model commonly adopted for product evaluation activities, but it has been recognized that specific emphasis on particular characteristics and subcharacteristics should be given depending on the product typology and the purpose of the evaluation. Many different quality profiles, as well as methods for their definition and for the selection of proper evaluation techniques, have been and are being provided for specific environments (e.g. software for banks).
Service addresses
users, customers
Service description
Methods for carrying out comparisons between software products typically consist of the following steps:
Example of class characteristics (step 1):
The following functional and non-functional characteristics could be defined for a set of testing tools:
| functional | non functional |
| - test planning and management | - usability |
| - test case generation | - environment |
| - coverage analysis | - price |
| - test development | |
| - test execution | |
| - simulation and load test |
Example of evaluation and comparison results (steps 4 and 5):
Service addresses
customers, suppliers, (users, certification bodies, test laboratories)
Service description
The typical situation demanding software product reengineering is when we have a product that has already been completely (or almost completely) developed without the (or with only a partial) application of any development standard and/or formal documentation and we need to restructure and re-document it.
The usual intervention activities and outputs can be expressed as shown in Fig. 1.
Fig. 1 - Re-engineering activities
The analysis of the structure and architecture of the existing code is usually accomplished by using specific tools performing static and dynamic code analysis.
Examples of reports produced by these tools are provided in fig. 2, 3, and 4.
The indications resulting from the analysis activities as well as the adoption of specific suitable development and documentation standards lead to the planning and the further execution of the re-engineering intervention.
Fig. 2 - Graphic report of code static analysis
Fig. 3 - Graphic report of code static analysis
Fig. 4 - Graphic report of code static analysis
Service addresses
customers
Service description
The monitoring process is intended to carry out the verification of requirements stated in a contract and/or agreed by a Customer and a Supplier (either external or internal).
Monitoring comprises all those activities that contribute to the achievement of the best possible result and of both the formal and substantial fulfilment of the requirements, thus guaranteeing the achievement of the qualitative and quantitative results planned for the project.
Monitoring aims to allow any Body or Company, that requires the support of a third party for the supply of IT application and/or services, to obtain substantial guarantees on the actual quality of the required product/service and, at the same time, the qualification of its Supplier through the evaluation of the Supplier's processes.
In brief, by means of the control and verification carried out by applying specific methodologies and techniques, monitoring guarantees:
The technique adopted for monitoring is oriented to the analysis and resolution of the detected problems rather than to the plain detection of faults and/or inefficiencies.
In short, the monitoring activities are intended to assure:
More details about the monitoring service can be found here.
