CMMMö/3.6.2002
Slide 1Efficiency Improvement of Software Development
ICEL
Quality Management Systems:Efficiency Improvement
in Software Development
Karl Heinrich MöllerGaertnerstr. 29
D-82194 GroebenzellTel: +49(8142)570144Fax: +49(8142)570145
Email: [email protected]
CMMMö/3.6.2002
Slide 2Efficiency Improvement of Software Development
ICEL
Contents
• Principles• Capability Maturity Model (CMM)
– Overview– Questions– Results
• Improvement Process– People within the improvement process– Organising the improvement process (example)– Charter of the groups engaged– Piloting the new methods– Application of the new methods
• Experience• Rules
CMMMö/3.6.2002
Slide 3Efficiency Improvement of Software Development
ICEL
PrinciplesVision
quality
productivity elapsed time
• derive objectives of the organisation• objectives achieved?
• who sets the objectives ?• who controls the achievement?• how is the achievement controlled?
CMMMö/3.6.2002
Slide 4Efficiency Improvement of Software Development
ICEL
org.-level 1org.-level 3
org.-level 1org.-level 2
employeeemployeeemployeeemployee
PrinciplesVision
organisation to be improved
employee
org.-level 1org.-level 1org.-level 1Change Agent
Change Agentchange agent
improvement project
project leader
sponsor
externalsupport
org.-level 4
CMMMö/3.6.2002
Slide 5Efficiency Improvement of Software Development
ICEL
PrinciplesVision
• PlanPlan the method
• DoIntroduce the method
• CheckCheck whether the goal has been achieved
• ActImprove the method
Plan
Act
Check
Do
CMMMö/3.6.2002
Slide 6Efficiency Improvement of Software Development
ICEL
Initial
Repeatable
Defined
Managed
Optimizing
Process is informal andad hoc
Project managementpractices areinstitutionalized
Technical practices areintegrated withmanagement practicesand institutionalized
Product and process arequantitatively controlled
Process improvementis institutionalized
Level Process Characteristics
Risk
Quality
Productivity
Results
Capability Maturity Model (CMM)
CMMMö/3.6.2002
Slide 7Efficiency Improvement of Software Development
ICEL
Maturity Level 1
• what has been achieved:every individual does its best to perform the work
• what is missing:cooperation of the individuals only if they believe there is a need
CMMMö/3.6.2002
Slide 8Efficiency Improvement of Software Development
ICEL
Maturity Level 2
• what has been achieved :the individuals of every project are organising their work to achieve the objectives of the project as far as they are understood
• what is missing:cooperation of the projects only if they believe there is a need (every project e.g. has its own configuration management system); processes are not efficient
CMMMö/3.6.2002
Slide 9Efficiency Improvement of Software Development
ICEL
Maturity Level 3
• what has been achieved :organisation has a defined development process valid for all projects, which is to be followed by everybody; process is optimised by qualitative criteria
• what is missing:no quantitative criteria (faults are found too late, fault prevention insufficient, many changes in late phases etc.)
CMMMö/3.6.2002
Slide 10Efficiency Improvement of Software Development
ICEL
Maturity Level 4
• what has been achieved :process is controlled and continuously improved by quantitative criteria
• what is missing:process improvements are endured and not created; introducing new methods is insufficiently planned and controlled
CMMMö/3.6.2002
Slide 11Efficiency Improvement of Software Development
ICEL
Maturity Level 5
• what has been achieved :introducing new methods is planned and controlled
• what is missing:nothing
CMMMö/3.6.2002
Slide 12Efficiency Improvement of Software Development
ICELSoftware Process AssessmentStructure of Questions (Clusters / Process Criteria)
Process Criteria
1.Organisation and
Resources
2.Process oriented
Functions
3. Life Cycle independent
Functions
4. Life Cycle Functions
1.1Organisationsal
structure
1.2
Training
1.3
Communication
1.4
Technology
2.1 ProcessDefinition
2.2 Processmeasurement
2.3 ProcessImprovement
3.1Project Planning
3.2
Project Control
3.4 ConfigurationManagement
3.3 Qualityassurance
3.5 SubsupplierManagement
3.6 Risk-Management
3.7 QualityManagement
4.6 Acceptance
4.7 Application andMaintenance
4.5 Integration andSystem Test
4.4 Implementation
4.3Design
4.2 RequirementManagement
Spec. SystemProperties
4.1
CMMMö/3.6.2002
Slide 13Efficiency Improvement of Software Development
ICELDevelopment Process: Conformance Diagram Site Assess. (I)
1 2 3 4 5
Organization
Training
Communication
Technologies Management
Process Definition
Process Measurement
Process Improvement
Project Planning
Project Tracing
Quality Assurance
Configuration Mgmt.
Contractor Mgmt.
Risk Management
Quality Management
CMMMö/3.6.2002
Slide 14Efficiency Improvement of Software Development
ICEL
Development Process: Conformance Diagram Site Assess. (II)
1 2 3 4 5
Customer Requirements
SW Requirements
Architecture Design
Detailed Design
Implementation
Integration
System Test
Acceptance Test
Deployment +Maintenance
CMMMö/3.6.2002
Slide 15Efficiency Improvement of Software Development
ICEL
Development Process: Conformance Diagram Project Assess. 11 2 3 4 5
Organization
Training
Communication
Process Definition
Process Measurement
Process Improvement
Project Planning
Project Tracing
Quality Assurance
Configuration Mgmt.
Contractor Mgmt.
Risk Management
Quality Management
Customer Requirements
SW Requirements
Architecture Design
Detailed Design
Implementation
Integration
System Test
Deployment + Maintenance
CMMMö/3.6.2002
Slide 16Efficiency Improvement of Software Development
ICEL
Improvement Process 1From Analysis to an Improved Status
Analysis Design Imple-mentation Test Use
identify 3 to 5 processes with the highest potential for improvement2
1
34
5
initial
repeatable
defined
managed
optimizing
method 1implementing
method 2implementing
method 4implementing
method 3implementing
method 1test
method 2test
method 3test
method 1use
CMMMö/3.6.2002
Slide 17Efficiency Improvement of Software Development
ICEL
Improvement Process 2 From Analysis to an Improved Status
Imple-mentation
Test Use
method 2implementing
method 3implementing
method 2test
method 3test
method 4implementing
method 4test
method 4use
method 1implementing
method 1test
method 1use
Check
Act
CMMMö/3.6.2002
Slide 18Efficiency Improvement of Software Development
ICEL
Roles within the Improvement Process
• SponsorHead of organisation
• Coachmembers of (another) organisation who make the change happen (full-time)
• Change Agentmembers of organisation who make the change happen (part-time)
• Championmembers of organisation who support the change
• Targetmembers of organisation whose attitudes have to be changed
CMMMö/3.6.2002
Slide 19Efficiency Improvement of Software Development
ICEL
Software Engineering Process Group Example
S ub-T eam 11R ev iew
S ub-T eam 12T es t
S ub-T eam 13D PP
Proc es s T eam 1D efec t E lim inat ion P roc es s
S ub-T eam 21A naly s is
S ub-T eam 22D es ign
S ub-T eam 23P roduc t P lann ing
P roc es s T eam 2Produc t C rea t ion P roc es s
S ub-T eam 31L ib P roduc t ion
S ub-T eam 32 P roduc t ion
S ub-T eam 33Q M -M anagem ent
P roc es s T eam 3 P roc es s C ont ro l
Sub-T eam 41M etric s
Sub-T eam 42C os t C ont ro l
P roc es s T eam 4P roc es s O rgan iz a t ion
SE P G
CMMMö/3.6.2002
Slide 20Efficiency Improvement of Software Development
ICEL
CharterSoftware Engineering Process Group SEPG
• Led by project leader• Supported by sponsor• Consisting of leaders of process teams• Controls improvement process
– Distribution of resources– Members (especially sponsor) stand for the project– Help in case of trouble– Decides on introduction of new methods
CMMMö/3.6.2002
Slide 21Efficiency Improvement of Software Development
ICEL
Charter Process Team
• Led by a member of SEPG• Controls sub teams• Consists of the members of the sub teams• Godfather supports introduction of new
methods
CMMMö/3.6.2002
Slide 22Efficiency Improvement of Software Development
ICEL
CharterSub Team (Technical Experts Group)
• Opinion leaders of the organisation• Designs, implements, tests, and introduces
new methods• Members become coaches and change agents
CMMMö/3.6.2002
Slide 23Efficiency Improvement of Software Development
ICEL
changeagent
Introduction of New Method 1Overview
technical experts group (subgroup) defines the new method
pilot 1 pilot 2
time
group a group b group c
Piloting
Introductionstep 1
Introductionstep 2
coach
proc
ess
team
con
trols
pilo
ting
and
intro
duct
ion
group d group e group f
CMMMö/3.6.2002
Slide 24Efficiency Improvement of Software Development
ICEL
org.-level 1org.-level 3
org.-level 1org.-level 2
employeeemployeeemployeeemployee
Introduction of New Method 2Technical experts team (Sub Team) designs it
organisation to be improved
employee
org.-level 1org.-level 1org.-level 1Change Agent
Change Agentchange agent
improvement project
project leader
sponsor
org.-level 4
CMMMö/3.6.2002
Slide 25Efficiency Improvement of Software Development
ICEL
org.-level 1org.-level 3
org.-level 1org.-level 2
employeeemployeeemployeeemployee
Introduction of New Method 3Piloting
organisation to be improved
employee
org.-level 1org.-level 1org.-level 1Change Agent
Change Agentchange agent
improvement project
project leader
sponsor
org.-level 4 actions:part 1Sub team gets pilot to support new methodpart 2pilot has been convinced that new method makes work more easy (and is of advantage to the organisation)
CMMMö/3.6.2002
Slide 26Efficiency Improvement of Software Development
ICEL
PilotingSteps
• Kick off meeting to join in members of the pilot• Members of pilot get every support• Measure degree of application and effectiveness• Final report together with the members of pilot
– Method makes work easy– Is really applied– Has advantage for the organisation
CMMMö/3.6.2002
Slide 27Efficiency Improvement of Software Development
ICEL
org.-level 3
org.-level 2
Introduction of New Method 4Convincing Management
organisation to be improved
employee
org.-level 1Change Agent
Change Agentchange agent
improvement project
project leader
sponsor
org.-level 4
action:improvement project convinces every level of management (especially middle management) of the advantages of new method
CMMMö/3.6.2002
Slide 28Efficiency Improvement of Software Development
ICEL
Rules for Cooperation 1Understanding the Improvement Project
• Without change of mental attitudes there is no acceptance of new methods
• Pressure is a bad means to enforce mental change
• To simply wait for understanding will not lead to a timely change
• To make a change happen real hard work with the people to be convinced is necessary
• Change will only occur with the majority of people in the organisation being in favour of change
CMMMö/3.6.2002
Slide 29Efficiency Improvement of Software Development
ICEL
Rules for Cooperation 2Rules of Enforcement
• Arguing until organisation knows nothing against• Every action must succeed the first time
(estimation of resistance must be accurate)– Coach or change agent is able to do it– Needs support from project leader or department head (should not
be done often) – Needs support of the sponsor (should not be done directly)
• Enforcement in the beginning must be superseded by trustful cooperation
CMMMö/3.6.2002
Slide 30Efficiency Improvement of Software Development
ICEL
Defect Detection RatesDesignRequire-
mentsLevel
Managed4
Defined3
Repeatable2
5% 10% 50% 20% 10% <5%
0% 0% 5% 15% 50% 30%
3% 5% 7% 25% 50% 10%
Cost per defect in DM
Average costper defect
3.700 DM
9.600 DM
12.900 DM
Imple-menting
Modul Test
Integr. Syst.-Test
Use
500 500 500 6.000 12.000 20.000
CMMMö/3.6.2002
Slide 31Efficiency Improvement of Software Development
ICEL
Potential of ImprovementExample for Fault Cost Reduction
Defect cost are a certain fraction of total development cost
There is an exponential increase in cost if time between seeding and detection becomes longer:
- detection of defects as early as possible- prevent defect - reduce number of undetected defects in
each phase of development process
Potentials for cost reduction:level 1 level 2 : 17 % of defect costlevel 2 level 3 : 22 % of defect costlevel 3 level 4 : 19 % of defect costlevel 4 level 5 : 44 % of defect cost
Defectcreation
rate
Defect detection
rate
Reaquire-ments Design Coding Functional
TestSystem
TestUse
Cost of defects(rates)
5
4
3
2
1
10%40% 50%
5% 20%40%
20%10% <5%
3% 5% 7%60%
20%5%
0% 2% 3% 55%32% 8%
0% 0% 3% 30%50%
17%
33%50%15%2%0%0%
1 11 1 6 1220
CMMMö/3.6.2002
Slide 32Efficiency Improvement of Software Development
ICELRelation of Maturity Level and Process CharacteristicsProduct Size: 500 KLOC
Highest gains for field defects and cost(Reported by Lockheed)
Initial
2
1
3
4
5
Repeatable
Defined
Managed
Optimizing
Elapsed Time[Time]
Field Defects[Defects/kLOC]
Productivity[LOC/hr]
ex-tra-po-la-tion
Cost[Cost/Project]
1%
3%
11%
22%
100%
9%
17%
33%
66%
100%
55%
65%
75%
85%
100%
100%
70%
40%
27%
20%
0 50 1000 50 1000 50 100 0 50 100
CMMMö/3.6.2002
Slide 33Efficiency Improvement of Software Development
ICEL
General experiences Effort and Elapsed Time
• 2 to 3 years are needed to improve projects by 1 maturity level
• 2 to 3% of development cost is a typical effort for process maturity of an organisation that
– is at SEI-Level 2 to 3 and– is insufficient to improve SEI-Maturity
• Up to 5% is needed to improve the maturity level
CMMMö/3.6.2002
Slide 34Efficiency Improvement of Software Development
ICEL
Gains for Improving from Maturity Level 2.5 to 3.5 Quality, Productivity and Elapsed Time
• Productivity is increased by about 50%• Elapsed time is reduced by about 15%• Field defect rate is cut by half
• Compared to 3 to 5% of effort for the process improvement