The history of the development of the quality management system of a leadingFinnish software house is briefly described paying attention to the premises ofsetting up a quality management system and the phases of its furtherdevelopment and relating the process to the total quality management approach.The case study history has led to a multilevel quality system where quality isingrained into all the major functions of a software house including softwareengineering, marketing and selling, personnel management, companymanagement and administration as well as quality control and developmentitself. The structure of the multilevel quality system is proposed as a generalstructure for a quality management system oriented towards software processimprovement.
Quality emerges into the functioning of a company basically in two ways: inits effects in customer satisfaction and in business results. The basic metrics formeasuring quality in the case study organization especially in terms of theseeffects are identified. Special attention is also paid to software processimprovement and the factors affecting it. Finally some remarks are made on theeffects of the quality management system on the identified company processes inthe case study. The company has been certified according to the ISO 9001standard in June 1992 as the first software house in Finland.
INTRODUCTION
What is quality? The concept may be divided into three main categories: productquality, quality of service and process quality (cf. Lillrank [15]). Taguchi [25]and Crosby [2] emphasize objectiveness and measurability when speaking aboutquality in general. Particularly Taguchi restricts quality only to production-
related, measurable issues with an ideal of uniform and controlled quality of aproduct. This approach may be interpreted as a traditional engineering scienceview on the subject. Deming [3] believes that quality can be achieved byconstantly improving the processes that produce services or products; whenprocesses get better, quality and productivity go up too. In this article we take adefinite process improvement oriented approach towards the development of aquality management system.
Software process quality and process maturity models as well asassessment procedures have received keen interest during the later years, startingfrom Crosby's [2] original idea of the 5 stages of quality and culminating at thepresent in the work of the Software Engineering Institute (SEI) on the CapabilityMaturity Model (CMM) (cf. e.g. Humphrey [8], [9], Paulk [18], Paulk et al[19], [20], [21], [22]), in the work of the Esprit/Bootstrap project (cf. e.g.Koch [14], Bicego et al [1], Jarvinen and Simila [13]), as well as in the work ofthe SPICE project (cf. e.g. Dorling [6]) in integrating and standardizing thevarious assessment and improvement approaches.
Quality has been approached also from the point of process standards andcertification, most notably in the international standard series ISO 9000framework (cf. ISO 9001 [11], ISO 9000-3 [12]), not to omit the otherapproaches (cf. e.g. DoD STD 2167A [5], and ESA [7]).
More holistic process oriented approaches include Deming's 14 Points forManagement (cf. Deming [3]), Total Quality Management (TQM) (cf. e.g.Zultner [26], Deutsch [4], and Shashkin and Kiser [24]), and ConcentratedQuality Management (cf. Moisala et al [17]).
In this treatise we concentrate on the fundamental development principles ofthe quality management system (QMS) as part of a general TQM approachbasing our ideas on the experiences of developing a particular operational QMSand using this QMS as an example repository.
TQM - A CASE STUDY HISTORY
The case study organization was established in 1985 as an independent softwarehouse. The company has grown during the almost ten years since itsestablishment into a respected software producing organization with a personnelof close to 80 systems designers and a steady customer list with domestic andexport references.
The internal development of the company was and is based on a simple twophase approach: 1. identification of development stage and 2. selection ofdevelopment steps.
The identification of development stage originally utilized the ideas ofCrosby [2]. Crosby's model of 5 stages of quality was modified within the
company and has been used widely in Finland in consultation work by thecompany since 1987. The modification is depicted in figure 1. Since 1992 thecompany has been a partner in the ESPRIT/Bootstrap project (cf. Bicego et al[1]) and the software maturity assessment and development approach developedin the project is presently being ingrained into the company processes.
"JUNGLE WAR" "PROJECT" WORKreq def
design
coding
testing
SOFTWAREPRODUCTIONMANAGEMENT
Figure 1: Identification of development stages.
Retrospectively the development of the company may be seen as theimplementation of several concurrent and parallel processes with a selectivepriority order depicted in figure 2. Each of these development steps was takenwith an explicit decision in the company directory board and with the explicitidea of "small continued steps towards improvement". The process may belikened to "kaizen" (cf. Imai [10]), i.e. gradual, unending improvement doinglittle things better, setting - and achieving - ever-higher standards, even thoughthe development decisions were strategic in nature. Just as well applicable isDemmg's fundamental principle: "Improve constantly and forever the systemdevelopment process to improve quality and productivity, thus constantlydecrease costs." (cf. Deming [3], p. 23). Not surprisingly the order of thedevelopment steps bears considerable similarity with the SEI maturity model (cfHumphrey [9]).
A MULTILEVEL AND MULTIPERSPECTIVE QMS
As Seghezzi [23] points out, management concepts which came on themarket during the last ten years do not fulfill the requirements of the new
paradigm. The new paradigm demands management concepts whichmultidimensional^ and concurrently consider the factors of quality, costs andtime. What is required is not a new independent quality management concept butan integration of quality management as a part of a total management concept.
development level
SELECTION OFDEVELOPMENTSTEPS
strategic & financial management
personnel & resource management
quality control
sales and marketing
project management & testing
time
Figure 2: Implementation of concurrent and parallel company developmentprocesses.
The main components of the multilevel and multiperspective QMS, which isproposed here and which strives towards fulfilling the requirements of the "newparadigm", include the following: quality policy, quality organization, qualitycontrol principles and levels as well as the implementation of the principles ateach level, and quality manual structure which facilitates the documentation andcommunication of the previous components in an effective manner. Thecomponents are built in or embedded into all the processes of the productionunit. The structure and form of the proposed QMS bears some similarity to themodel proposed by Seghezzi [23] but has been developed independently as partof the normal organizational development of the case study company. Thegeneral structure of the multilevel and multiperspective QMS is described infigures 3 and 4.
The general structure of the quality organization represents themultiperspective view of the proposed QMS. Quality is viewed from theperspectives of all the major organizational units of the company. Each viewproduces an independent, slightly different and complementary outlook onquality. The solution depicted in figure 3 is based on a software producingorganization where the production is organized within system groups. A
different organizational structure would require a slight adjustment but the basicsolution would remain the same.
The quality management levels and functions depicted in figure 4 representon the other hand the multilevel structure of the proposed QMS where thelowest level is formed by the projects and the highest by company management.
Quality function| Quality group || Quality manager |
| Quality council |
Projects[ Project group
| Project directory board
System groups
persons w customer resp |
[ Group managers |
Company managementManaging director ]Directory groups |1Resource group |
Financial fu
Office ser
nction
vices
Figure 3: General structure of quality organization.
Each of the components of the multilevel and multiperspective QMS is inthe following treated separately firstly on a general level and then providingconcrete examples of each using the case study as a repository.
Quality policy
The quality policy statement is defined by the top management of the companyand it expresses the general objectives and attitudes of the organization towardsquality. The quality policy should form an integral part of the business conceptof the company. The statement of the quality policy is not in itself sufficient,what is also needed is the implementation mechanism of the policy.
According to its business concept: CCC is a software house that providescustomized consulting and software engineering for both export and domesticmarkets, emphasizing quality and delivery on time. The business concept ofCCC includes the definition of its quality policy: CCC emphasizes in itssoftware production especially quality and delivery on time.
PROJECT LEVEL FUNCTIONS
Project Projectmanagement
Technicalproject
Testingreviews
1SYSTEM GROUP LEVEL FUNCTIONS
Projectsales ContractreviewsProject imple-mentation and
qualityfollow-up
Personnel,financial,resource, and
interest groupmanagement
1COMPANY LEVEL FUNCTIONS
Managementand qualityfollow-up
Qualitysystemdevelopment
Qualitysystemreviews and
auditsPersonneltraining
Figure 4: Quality management levels and functions.
Delivery on time means that the system ordered by the customer satisfies thequality objectives and is delivered in the agreed schedule and in the given frameof personnel, facilities, equipment and software resources. The quality ofsoftware production means that the customer gets the system that he/she needsand that the system satisfies the requirements of the customer. This kind ofsystem is useful for the customer organization, it satisfies the end users, it isreliable and it is also technically effective from the viewpoint of both use andfurther development.
The implementation of the quality policy is based on the strong projectculture of CCC. Quality awareness of the personnel is the most central factor inthe project culture at the individual level. Quality awareness means that eachperson is responsible for the implementation of the quality policy of thecompany and performs the tasks according to the given instructions and attemptsalso to develop his/her own working community. In other words everyoneparticipates in the implementation of the quality policy and in the furtherdevelopment of the quality of CCC.
Quality awareness requires that the personnel has very good project deliveryknow-how, customer business process know-how, information technologyknow-how, cooperation skills as well as self-development desire and result-orientation. At the activity level the systematicness of procedures is the centralfactor of project culture. This means that all tasks are performed in a planned andconsistent manner using appropriate methods as well as techniques and toolssupporting their use.
The development of procedures takes place through cooperation between allthe personnel and is led by the quality function. The central objective of thedevelopment is to increase the systematicness by decreasing rework andoverlapping work and so to improve work productivity. The rework isdecreased when error preventive design methods are used and sufficientinvestment is made for an effective testing procedure covering all the workphases with the objective of finding the errors as early as possible. Theoverlapping work is decreased when the work practices and methods of theorganization are such that it can utilize previously done work as much aspossible. CCC has a project data system that implements this principle andsupports the collection of experience data and different solutions.
In the development of the quality system CCC has applied total qualitythinking, which means that the whole company is an object of development.This means also that the whole organization of CCC is a quality organization.
Quality organization
The quality organization defines the organizational structure, responsibilities,procedures, and resources for the implementation of quality management.
The responsibilities of the units of the quality organization concern thedifferent levels of the company, i.e. company management, the quality functionitself, system groups, and projects.
Company management The responsibilities of company management withrespect to quality are the following: to determine the quality policy, to review thequality system regularly, to perform quality monitoring at the company level,and to set the resource framework for the quality system
Quality function Within the quality function itself the responsibilities are dividedbetween the quality council, the quality manager, and the quality group. Thequality council determines the contents of the quality system that implements thequality policy of the company, performs internal quality audits regularly, andmanages the development of the quality system. The quality manager isresponsible for the development and maintenance of the quality system andimplementation of the quality assurance functions in software production. Thequality group performs quality assurance work and quality monitoring at theproject and system group level, participates in the development of the qualitysystem, and gives quality support and education to the system groups
System groups Within the system groups the responsibilities with respect toquality are divided between the group manager, the person with business arearesponsibility, and the person with customer responsibility. The group manageris responsible for the implementation of the projects of the group according tothe quality system and responsible for the personnel management of the group.The person with business area responsibility is responsible for the businessactivity on the business area, performs quality monitoring of the project andwhen needed makes corrective actions. The person with customer responsibilityis responsible for specified customer relationships and the quality of thecooperation
Projects Within the projects the responsibilities are divided between the projectdirectory board, the project manager, project group, and the test support person.The project directory board performs the quality monitoring of the project, andmakes decisions concerning the changes of the project contract. The projectmanager is responsible for the carrying out of the project according to the plansand decisions of the directory board, performs quality assurance according to thequality system within the project, makes the project plan which includes also thequality plan that defines the quality practices in the project, and gives feedbackabout the quality system. The, project group is responsible for the progress of theproject according to the plan, implements the quality policy by performing theproject tasks according to the given guidelines, and gives feedback about thequality system. The test support person is responsible for the planning andimplementation of testing together with project group.
Quality control principles and levelsThe central principle of the quality control is that quality is created by making -not by monitoring. The projects and groups are responsible for the production ofquality. The task of the quality function is to support the projects and systemgroups so that they are able to reach their quality objectives.
CCC's quality function is clearly a support activity by its nature. On theother hand it is also mainly an error prevention activity even though its task isalso to react immediately to the situations requiring error corrections. From theviewpoint of the customer the quality function offers a channel which he/she canuse to express his/her problems and hopes directly to the management ofsoftware production.
The quality control takes place in CCC at three levels: at the company level,system group level and project level.
Quality control at the company level At the company level the quality controlconcentrates on the evaluation and development of the quality system and thesoftware production process, the implementation of internal quality educationand on the quality monitoring concerning the company level.
The management reviews and evaluates the quality system yearly from theviewpoint of the business concept. The quality council makes quality auditsevery six months and evaluates the contents of the quality system Theevaluation is based on the results of the quality monitoring that are the reports ofquality function, customer and personnel feedback as well as the key figures ofthe quality measures.
The development takes place under the control of the quality council and bythe quality manager and quality group in cooperation with all the personnel Theguidelines of development are set in the yearly plan of the quality function Themanagement follows the quality of the projects continually on the basis of thereports of the projects and the quality group and makes corrective actions ifneeded.
Group level quality rontrol At the system group level the main areas of qualitycontrol are personnel, economy, resource and interest group managementmanagement of sales projects as well as the implementation of project andquality assurance. The activities are based on the yearly plan of the systemgroup. ^
The development of quality awareness and the professional skills of the
** ™°* ™P°"*" tasks of personnel management at the system
The success of projects requires that the projects have the right resourcesine group manager is responsible for the resources of the projects The resourcegroup co-ordinates the resource allocation between the system groups Thesystem groups are responsible for the contracting and implementation of theprojects. From the viewpoint of the quality, continuous quality monitoring andquick corrective actions are important.
Project level quality control The quality control at the project level concerns thequality ot the production process and that of the product.
The quality of the project is sufficient, when it can be carried out in thegiven frame of resources and when it is possible during the project to developcooperation between the different interest groups according to the objectives putforward in the project plan. The quality of the product is sufficient, when itsatisfies the requirements of the customer. The product must be useful to theorganization of the customer, it must satisfy the end users, and it has to bereliable and technically effective from the viewpoint of both use and furtherDevelopment.
. , *e Project level is divided into two parts: project control andthe technical project that includes depending on the project: feasibility studyspecification, design, implementation, and testing phases as well as installationA II (I mnint"£»nanr»£»and maintenance.
The quality assurance is also divided into two parts: the internal and theexternal quality assurance. They concern the quality of both the product and theproject. The project is itself responsible for the internal quality assurance. Thequality manager and quality group are responsible for the external qualityassurance. The customer may also be responsible for the external qualityassurance depending on the contract.
The internal quality assurance of the product takes place according to theaccepted testing plans and the internal quality assurance of the project takes placein the project meetings led by the project manager. The test support persontogether with project group are responsible for the preparation of testing plans.The external quality assurance of the product takes place in the testing reviewsand of the project in the project reviews.
Quality manualThe quality manual documents the quality system of the company and alsoprovides the multiperspective view on quality required for its implementation.The quality manual of CCC follows the ISO 9001 and 9000-3 standards.
The quality manual consists of 9 parts: I General description of the qualitysystem, II Quality instructions for company management, III Qualityinstructions for system group, IV Quality instructions for project control, VQuality instructions for design and testing, VI Instructions for the qualityfunction, VII Instructions for the economy function, VIE Instructions for officeservices, and IX General instructions.
QUALITY METRICS APPROACH - BASIC DATA FOR PROCESSIMPROVEMENT
The main measurements for measuring quality in the case study are: 1) customersatisfaction profile of a project and over the projects, 2) project productivityprofile and summary profile over the projects, 3) project quality profile andsummary profile over the projects, and 4) employee satisfaction profile onquality system and the results of supporting functions of the company [cf. e.g.the criteria of the European Quality Award].
Customer satisfaction profileThe customer satisfaction profile includes the following measures: satisfactionon the project, satisfaction on the results of the project, satisfaction on theservices of the project.
The customer satisfaction measure at the project level is based on actualdata gathered from the customers of the project using customer questionnaires.The main factors that customers are asked to evaluate include among other thingsdelivered products, co-operation, reporting, working practices, professionalskills of workers and their co-operation capabilities. These factors and their subfactors are evaluated using a five value semantic differential. At the group level
the feedback of customers is summarised by calculating the numbers of thedifferent values given by the customers to the questions in the questionnaires,over the projects and different time periods within the group and by presentingthe results as summary and scenario diagrams. At the com/xwy 7eW the samekind of summary data are produced from the group level data.
Project productivity profilp
The productivity profile consists of six components: 1) the difference betweenthe planned and real amount of work, 2) the share of rework, 3-5) theeffectiveness of specification, design and implementation testing, and 6) thequality index of the product ex post.
The basic data are gathered using work amount, and error and correctionwork amount matrixes. In the work amount matrix we have for each activity theplanned and real amounts of work. In the error and correction work amountmatrix we have for each phase of testing (i.e. from the specification phase to thesystem testing phase and use phase during the warranty time) the amounts oferrors found and amount of work used for the correction of different types oferrors (i.e. specification, design and implementation errors). Using this data wecan determine the components of the profile. The rework of a project is the sumof error correction work. The effectiveness of (specification, design andimplementation) testing is the percentile share of errors found in the phase inquestion (specification, design, implementation) compared to all the errors ofthe same type found during the whole life cycle of the system up to the end ofthe warranty time. The quality index of the product is the percentile share of theerrors found during the development process compared to all the errors found upto the end of the warranty time.
The differences and the share of rework are classified into five categoriesvalued 1-5: 1= over 20 %, 2= 16-20 %, 3= 11-15 %, 4= 6-10 %, and 5= 0-5%. The effectiveness values and indexes are also classified into five categoriesvalued 1-5: 1= under 60 %, 2= 61-70 %, 3= 71-80 %, 4= 81-90 % and 5= 91-100 %.
At the group and company level the productivity profiles are determined bycalculating the numbers of projects belonging to the different classes accordingto the components of the project productivity profiles over the groups and/ordifferent time periods and presenting the results as summary and scenariodiagrams. The productivity profiles are supplemented by economical results(e.g. project net profit). An example of project productivity profiles is providedin figure 5.
Project quality profile
The project quality profile is based on the subjective assessments made by thequality group on how well the instructions of the quality manual of the companyare followed in the projects using a five value semantic differential. Things
evaluated are such as project planning, project control, reporting, risk andconfiguration management, and testing. Based on the values above the qualityindex is calculated for the project as a weighted average. At the group level thequality index for the group is determined as an average of the quality indexes ofthe projects of the group and the company level the quality index is an averageof the group indexes.
Project productivity profiles in the system group 2
Project 1 Project 2 Project 3 Project 4 Project 5
I Planned vs actualeffort
O Share of rework 3 Effect of specstesting
• Effect, of design E3 Effect, of impl. testing Q Quality index oftesting product
Figure 5: An example of project productivity profiles.
Employee satisfaction profileThe employee satisfaction profile is based on the following measures:satisfaction on quality function operations (e.g. project and testing reviews,methodology support, quality training, quality instruction manual), satisfactionon personnel management operations, satisfaction on office services, andsatisfaction on economical services. The employee satisfaction data are gatheredwith the aid of the questionnaires concerning each topic in question and using afive value semantic differential. The individual opinions are summarised bycalculating averages and modes for all the factors and presented as summary andscenario diagrams.
EFFECTS OF QMS IN TERMS OF COMPANY PROCESSES
Even though we have not yet made any thorough formal evaluation of the effectsof the developed QMS on the business processes of the company, a quite strongindication of the effectiveness of the soundness of the proposed andimplemented approach is provided by the business position of the company. Thecompany has been growing steadily both in terms of revenue and personnelthrough the most difficult years in recent history in the Finnish economical life.The present depression seems to have only strengthened the business position ofthe company.
1. Bicego, A., Cacchia, R., Haase, V., Koch, G., Krzanik, L., Kuvaja,P., Lancelotti, R., Maiocchi, M., Messnarz, R., Saukkonen, S. Schynoll, W.]Simila, J. 'Bootstrap: Europe's Assessment Method.' IEEE Software pp 93-95, May, 1993.
2. Crosby, P. B. Gwa% ^ Frgg - 7Ag ArfMcGraw-Hill, 1979.
3 . Deming, W. E. Out of the Crisis, MIT Center for Advanced EngineeringStudy, Cambridge, MA, 1986.
4. Deutsch, M. S. Total Quality Management in Hughes Aircraft',Proceedings of the Esprit Bootstrap Conference on Lean Software DevelopmentStuttgart, October 22-23, 1992.
5. DOD-STD-2167A, 'Military Standard DOD-STD-2167A', DefenceSystem Software Development, February 29, 1988.
6. Dorling, A. 'SPICE: Software process Improvement and CapabilitydEtermination.' Information and Software Technology, Vol 35 No 6/7 nn404-406, 1993. ' "
7. ESA, ESA (European Space Agency) Software Engineering StandardsESA PSS-05-0 Issue 2, February, 1991.
8. Humphrey, W. S. 'Characterizing the Software Process' IEEESoftware, Vol. 5, No. 2, March, pp. 73-79, 1988.
9. Humphrey, W. S. Managing the Software Process, Addison- WesleyReading, MA, 1989.
10. Imai, M. Kaizen: The Key to Japan's Competitive Success, McGraw-Hill, New York, 1986.
11. ISO 9001, Quality Systems; Model for Quality Assurance inDesign/Development, Production, Installation and Servicing, 1987.
12. ISO 9000-3, Quality Management and Quality Assurance Standards, Part3: Guidelines of ISO 9001 to the Development, Supply and Maintenance ofSoftware, 1991.
13. Jarvinen, J. and Simila, J. 'Maturity Level Definition: A ComparativeAnalysis.' in Proceedings of the 16th IRIS, (ed. Bansler, J. P., Bodker, K.,Kensing, F., Norbjerg, J., and Pries-Heje, J.), Information systems Researchseminar In Scandinavia, 7-10 August, 1993, Rapport Nr. 93/16, Department ofComputer Science, University of Copenhagen, 1993.
14. Koch, G. R. 'Process Assessment: the 'Bootstrap' Approach',Information and Software Technology, Vol. 35, No. 6/7, pp. 387-403, 1993.
15. Lillrank, P. Laatumaa (Land of Quality), Gaudeamus, Jyvaskyla, 1990.
16. Manns, T. and Coleman, P. Software Quality Assurance, MacMillan,Basingstoke, 1988.
17. Moisala, U. M., Vuorinen, R. and Moisala, A. CQM tyoyhteisonmuuttajana (Changing Work Community with CQM), Weilin+Goos,Hameenlinna, 1989.
18. Paulk, M. C. 'U.S. Quality advances: The SEI's Capability MaturityModel', Proceedings of 3rd European Conference on Software Quality, Madrid,November 3-11, 1992.
19. Paulk, M. C., Curtis, B., Chassis, M. B. et al, Capability MaturityModel for Software, Software Engineering Institute, CMU/SEI-91-TR-24,DTIC Number ADA240603, August, 1991.
20. Paulk, M. C., Curtis, B., Chrissis, M. B. and Weber C. V., 'CapabilityMaturity Model, Version 1.1', IEEE Software, pp. 18-27, July, 1993.
21. Paulk, M. C. et al, Capability Maturity Model for Software, VersionLI, Software Engineering Institute, Technical Report CMU/SEI-93-TR24,1993.
22. Paulk, M. C. et al, Key Practices of the Capability Maturity Model,Version LI, Software Engineering Institute, Technical Report CMU/SEI-93-TR25, 1993.
23. Seghezzi, H. D. 'Europe as Part of the Triad', in EOQ 93 (Ed. Anttila,J.), pp. 12-25, Proceedings of EOQ '93 World Quality Congress, Helsinki,Finland, June 15-17, 1993. Salpausselan Kirjapaino, Finland, 1993.
24. Shashkin, M. and Kiser, K. J. Putting Total Quality to Work, Berret-Koehler, San Fransisco, 1993.
25. Taguchi, G. Introduction to Quality Engineering, Asian ProductivityOrganization, Tokyo, 1986.
26. Zultner, R. E. 'Software Quality Engineering: The Deming Way',American Programmer, Vol. 3, No. 11, pp. 2-11, 1990.
