Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 1© 2004 by Carnegie Mellon University
Software Engineering InstituteCarnegie Mellon University
Process Maturity Profile
Software CMM®
2003 Year End Update
March 2004
We could not produce this report without the support of the organizations and lead appraisers who reported their appraisal results to the SEISM.
Our many thanks for their continuing cooperation with our data collection and analysis efforts.
Software Engineering Measurement and Analysis Initiative
The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Def ense and operated by Carnegie Mellon® University
® CMM and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon UniversitySM SEI is a service mark of Carnegie Mellon University
© 2004 by Carnegie Mellon University
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 2© 2004 by Carnegie Mellon University
OutlineIntroduction
Current Status
Community Trends
Organizational Trends
Summary
This briefing is based solely on the number of appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using SEI models or appraisal methods nor can it be used for certification or verification purposes.
The SEI does not certify organizations. The SEI only licenses and authorizes lead appraisers to conduct appraisals. Neither the SEI nor any other organization is a “certifying authority” of the results from an appraisal. Therefore requests to the SEI to provi de information to be used for this purpose can not be fulfilled. Information provided by the SEI, such as this briefing, is to demonstrate use of its products and services only.
Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/managing/app.directory.html
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 3© 2004 by Carnegie Mellon University
Introduction - 1: Purpose and SourceCharacterize the process maturity of the software community
This briefing uses information from appraisals using the SW-CMM in:
• CMM®-Based Appraisals for Internal Process Improvement (CBA IPIs) and
• Software Process Assessments (SPAs) and
• Standard CMMI® Appraisal Method for Process Improvement (SCAMPI), v1.1
Please visit:
http://www.sei.cmu.edu/sema/profile_about.html
for additional information or to answer questions you may have about this briefing
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 4© 2004 by Carnegie Mellon University
Introduction - 2: Briefing ContentsThis briefing includes three primary sections:
1. Current Status- Snapshot of the software community based on the most
recent appraisal, since , of reporting organizations
2. Community Trends- Global distribution of appraisals- Growth in the number of appraisals performed - Shifts in the maturity profile over time
3. Organizational Trends- Analysis of Key Process Area (KPA) satisfaction - Time to move up in maturity
1999
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 5© 2004 by Carnegie Mellon University
Section 1: Current StatusAppraisals conducted from throughand reported to the SEI by
• organizations• participating companies• projects• Non-USA organizations
Please refer to: Terms Used in this Report on page 29
1,593686
8,18554.7%
December 20031999January 2004
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 6© 2004 by Carnegie Mellon University
Based on most recent appraisal, since , of organizations reporting a maturity level rating
Maturity Profile by All Reporting Organizations
% o
f Org
aniz
atio
ns
1,5931999
12.3%
43.3%
27.4%
7.8% 9.2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Initial Repeatable Defined Managed Optimizing
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 7© 2004 by Carnegie Mellon University
Based on organizations reporting organization categories
Reporting Organization Categories
3.7%
20.5%
75.8%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Military/Federal
DoD/Fed Contractor
Commercial/In-house
% of Organizations
1,593
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 8© 2004 by Carnegie Mellon University
Based on most recent appraisal, since , of
Maturity Profile by Reporting Organization Categories
% o
f Org
aniz
atio
ns
1999 1,593
12.4%
45.0%
23.8%
8.4%10.3%9.5%
35.3%
42.9%
5.8% 6.4%
25.4%
50.8%
15.3%
6.8%
1.7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Initial Repeatable Defined Managed Optimizing
Commercial/In-house DoD/Fed Contractor Military/Federal
organizations reporting categories and a maturity level rating
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 9© 2004 by Carnegie Mellon University
Based on organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html
Business Services35.8%
Instruments And Related Products
6.1%
Transportation Equipment6.0%
Electronic & Other Electric Equipment
5.4%
Industrial Machinery And Equipment
4.9%
Retail Trade0.8%Wholesale Trade
0.7%
Transportation, Communication, Electric, Gas
and Sanitary Services6.5%
Public Administration (Including Defense)
8.0%
Engineering & Management Services10.8%
Services, Nec0.3%
Health Services0.3%
Finance, Insurance and Real Estate12.1%
Furniture And Fixtures0.1%
Food And Kindred Products0.1%
Fabricated Metal Products1.1%
Primary Metal Industries0.3%
Printing And Publishing0.5%
Chemicals And Allied Products0.3%
Organization TypeBased on Primary Standard Industrial Classification (SIC) Code
Manufacturing25.1%
Services47.4%
738
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 10© 2004 by Carnegie Mellon University
2000+1.0%
1001 to 20001.9%
501 to 10004.9%
301 to 5008.2%
26 to 5017.8%
51 to 7512.5%
76 to 1009.3%
201 to 30010.0%
101 to 20022.4%
25 or fewer12.0%
Based on organizations reporting size data
Organization SizeBased on the total number of employees within the area of the organization that was appraised
201 to 2000+26.0%
1 to 10051.3%
1,561
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 11© 2004 by Carnegie Mellon University
15.4
%
14.0
%
12.8
%
14.5
%
10.6
%
10.3
%
9.4%
5.3%
13.3
%
12.5
%
66.5
%
54.3
%
45.6
%
39.3
%
37.8
%
37.8
%
32.0
%
23.7
%
13.3
% 18.8
%
15.4
%
23.0
% 28.7
%
26.2
%
33.2
%
27.6
%
34.4
%
35.5
%
33.3
%
25.0
%
1.1%
5.4%
5.6%
11.0
%
10.3
%
9.0%
7.8%
14.5
%
10.0
%
12.5
%
1.6% 3.2%
7.2% 9.
0%
8.0%
15.4
%
16.4
% 21.1
%
30.0
%
31.3
%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+
Initial Repeatable Defined Managed Optimizing
Maturity Profile by Organization SizeBased on the total number of employees within the area of the organization that was appraised
Based on organizations reporting size data and a maturity level rating
The number of organizations for the 2000+ category is small whic h inflates the maturity level bars. Please see page 9 and take this into account. The purpose of this chart is to indicate that all sizecategories contain most, if not all, maturity levels.
1,561
% o
f Org
aniz
atio
ns
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 12© 2004 by Carnegie Mellon University
USA and Non-USAReporting Organization Categories
5.7%
49.0%
3.7%
14.8%
26.8%
0% 10% 20% 30% 40% 50% 60%
Military/Federal
DoD/Fed Contractor
Commercial/In-house
% of Organizations
USA
Non-USA
Based on U.S. organizations and Non-USA organizations reporting organization categories872721
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 13© 2004 by Carnegie Mellon University
Based on U.S. organizations and Non-USA organizations reporting categories and a maturity level rating
Maturity Profile by All Reporting USA and Non-USA Organizations
% o
f Org
aniz
atio
ns
18.4%
48.0%
26.1%
4.2% 3.3%7.2%
39.3%
28.6%
10.8%14.1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Initial Repeatable Defined Managed Optimizing
USA Non-USA
872721
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 14© 2004 by Carnegie Mellon University
CBA IPIs , SPAs and SCAMPI v1.1 using the SW-CMM conducted since 1987 through and reported to the SEI by
• appraisalsCBA IPIsSPAsSCAMPI
• organizations• participating companies• reappraised organizations• projects
Please refer to: Terms Used in this Report on page 29
Section 2: Community Trends
December 2003January 2004
3,1572,663
2,401829598
13,266
48410
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 15© 2004 by Carnegie Mellon University
Countries Where Appraisals have been Performed and Reported to the SEI
Argentina Australia Austria Barbados Belgium Brazil Canada ChileChina Colombia Costa Rica Cyprus Denmark Egypt Finland FranceGermany Greece Hong Kong Hungary India Ireland Israel ItalyJapan Korea, Republic of Latvia Malaysia Mexico Netherlands New Zealand NorwayPakistan Peru Philippines Poland Portugal Puerto Rico Russia Saudi ArabiaSingapore South Africa Spain Sweden Switzerland Taiwan Thailand TurkeyUkraine United Arab Emirates United Kingdom United States Uruguay Venezuela Viet nam
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 16© 2004 by Carnegie Mellon University
Number of Appraisals and Maturity Levels Reported to the SEI by Country
The organization maturity profile, page 8, can be created for certain countries using the Interactive Maturity Profile availablein the Software Engineering Information Repository Web site. Please visit: http://seir.sei.cmu.edu. If you are not already a member of the SEIR web site, please register to join. Registration and use of the site is free.
CountryNumber of Appraisals
Maturity Level 1
Reported
Maturity Level 2
Reported
Maturity Level 3
Reported
Maturity Level 4
Reported
Maturity Level 5
Reported CountryNumber of Appraisals
Maturity Level 1
Reported
Maturity Level 2
Reported
Maturity Level 3
Reported
Maturity Level 4
Reported
Maturity Level 5
ReportedArgentina 10 or fewer Malaysia 10 or fewerAustralia 34 Yes Yes Yes No Yes Mexico 22 Yes Yes Yes Yes YesAustria 10 or fewer Netherlands 21 Yes Yes Yes No YesBarbados 10 or fewer New Zealand 10 or fewerBelgium 10 Norway 10 or fewerBrazil 23 Yes Yes Yes Yes No Pakistan 10 or fewerCanada 67 Yes Yes Yes No Yes Peru 10 or fewerChile 12 Yes Yes Yes No No Philippines 10China 152 Yes Yes Yes Yes Yes Poland 10 or fewerColombia 10 or fewer Portugal 10 or fewerCosta Rica 10 or fewer Puerto Rico 10 or fewerCyprus 10 or fewer Russia 10 or fewerDenmark 10 or fewer Saudi Arabia 10 or fewerEgypt 10 or fewer Singapore 18 Yes Yes Yes No YesFinland 10 or fewer South Africa 10 or fewerFrance 124 Yes Yes Yes Yes Yes Spain 13 No Yes Yes No YesGermany 49 Yes Yes Yes No No Sweden 10 or fewerGreece 10 or fewer Switzerland 10 or fewerHong Kong 10 or fewer Taiwan 10 or fewerHungary 10 or fewer Thailand 13 Yes Yes Yes No NoIndia 330 Yes Yes Yes Yes Yes Turkey 10 or fewerIreland 10 or fewer Ukraine 10 or fewer
Israel 30 Yes Yes Yes No NoUnited Arab Emirates 10 or fewer
Italy 30 Yes Yes Yes No Yes United Kingdom 133 Yes Yes Yes No NoJapan 113 Yes Yes Yes Yes Yes United States 1838 Yes Yes Yes Yes Yes
Korea, Republic of 60 Yes Yes Yes Yes Yes Uruguay 10 or fewerLatvia 10 or fewer Venezuela 10 or fewer
Vietnam 10 or fewer
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 17© 2004 by Carnegie Mellon University
Based on appraisals conducted through and reported to the SEI by
Number of Appraisals Reported to the SEIby Year
0
100
200
300
400
500
600
1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
First Appraisal Reappraisal
3,157 Dec. 2003 Jan 2004
*
*
Num
ber
of A
ppra
isal
s
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 18© 2004 by Carnegie Mellon University
Based on appraisals reporting organization categories
Number of Appraisals Reported to the SEIby Organization Categories and Year
0
50
100
150
200
250
300
350
400
450
1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
Commercial/In-house DoD/Fed Contractor Military/Federal
Num
ber
of A
ppra
isal
s
3,155
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 19© 2004 by Carnegie Mellon University
80.3
%
12.1
%
7.6%
64.4
%
21.9
%
12.0
%
1.4%
0.3
%
37.3
%
33.9
%
20.2
%
4.5%
4.2%
32.2
% 35.7
%
20.9
%
5.9%
5.3%
27.7
%
37.1
%
22.8
%
6.0%
6.4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Initial Repeatable Defined Managed Optimizing
1987-19911992-19961997-200120022003
Trends in the Community Maturity Profile%
of O
rgan
izat
ions
Based on a cumulative view of the most recent appraisals of organizations up through the year indicated. This accounts for the difference from the figures on page 8.
Year Orgs132634
1,6601,9962,401
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 20© 2004 by Carnegie Mellon University
Section 3: Organizational TrendsAppraisals conducted through and reported to the SEI by
• Key Process Area (KPA) profiles- satisfaction of KPAs by maturity level for organizations
appraised at levels 1 and 2
• reappraised organizations- accounting for appraisals- although some organizations conducted multiple
reappraisals, only the first and latest appraisals were used in creating the charts on pages 23 & 24
Please refer to: Terms Used in this Report on page 29
598
1,656
December 2003
1,364
January 2004
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 21© 2004 by Carnegie Mellon University
Based on CBA IPI and SCAMPI with SW-CMM appraisals reporting a maturity level rating
Key Process Area Profiles - 1Organizations Appraised at Level 1
% of Appraisals
562
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
PCM
TCM
DP
SQM
QPM
PR
IC
SPE
ISM
TP
OPD
OPF
SCM
SQA
SSM
PTO
SPP
RM
FullySatisfiedRated
Repeatable
Defined
Managed
Optimizing
Software Subcontract Management (SSM) is not applicable/not rated in many assessments. Please take that into account when interpretingits Fully Satisfied rating.
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 22© 2004 by Carnegie Mellon University
Key Process Area Profiles - 2Organizations Appraised at Level 2
Based on CBA IPI and SCAMPI With SW-CMM appraisals reporting a maturity level rating1094
% of Appraisals
Defined
Managed
Optimizing
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
PCM
TCM
DP
SQM
QPM
PR
IC
SPE
ISM
TP
OPD
OPF
FullySatisfied
Rated
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 23© 2004 by Carnegie Mellon University
Maturity Level of First and Latest Appraisal
% o
f Org
aniz
atio
ns
43.6%
38.0%
10.0%7.7%
0.7%
7.7%
27.3%
41.1%
8.0%
15.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Initial Repeatable Defined Managed Optimizing
First Latest
Based on reappraised organizations using their first and latest appraisal598
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 24© 2004 by Carnegie Mellon University
No Change12.0%
Moved Down2.7%
Level 4 to 57.0%
Level 3 to 55.4%
Level 3 to 42.8%
Level 1 to 41.8%
Level 1 to 312.5%
Level 2 to 52.2%
Level 1 to 222.7%
Level 2 to 42.8%
Level 1 to 50.8%
Level 2 to 327.1%
ReappraisalsChange in Maturity Level
Based on reappraised organizations using their first and latest appraisal598
Level 1 to 1 5.7%Level 2 to 2 4.2%Level 3 to 3 1.2%Level 4 to 4 0.5%Level 5 to 5 0.5%
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 25© 2004 by Carnegie Mellon University
Time to Move Up
Number of months to move to nextmaturity level
Largest observed value that is not an outlier
75th Percentile
25th PercentileMedian
Smallest observed value that is not an outlier
Recommended time between appraisals
{
100
30
18
0
50
All (1987 to Present)
1 to 2 2 to 3 3 to 4 4 to 5188 234 49 55
1992 to Present
1 to 2 2 to 3 3 to 4 4 to 5163 222 47 55
Pre-1992
1 to 2 2 to 325 12
75
2224
39 40
Time Period of Initial Appraisal
LevelOrgs
25
19 2025
13 13
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 26© 2004 by Carnegie Mellon University
Summary of Current Status
Commercial and in-house organizations reporting most appraisals
Of U.S. organizations, the services and manufacturing industries reported most appraisals
Half of the organizations reporting size have 100 or fewer software personnel
More Non-USA appraisals reported than U.S. in this 5 year snapshot
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 27© 2004 by Carnegie Mellon University
Summary of Organizational Trends - 1
Software Quality Assurance is the least frequently satisfied level 2 KPA among organizations* appraised at level 1
Integrated Software Management is the least frequently satisfiedlevel 3 KPA among organizations* appraised at level 2
Higher maturity has been reached among those organizations reporting reappraisals
*Adjusted for number of organizations rating the KPA
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 28© 2004 by Carnegie Mellon University
Summary of Organizational Trends (continued) - 2
For organizations that began their CMM-based SPI effort in 1992 or later, the median time to move from:
• maturity level 1 to 2 is 22 months• maturity level 2 to 3 is 19 months• maturity level 3 to 4 is 25 months• maturity level 4 to 5 is 13 months
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 29© 2004 by Carnegie Mellon University
Terms Used in this ReportAppraisal - The appraisal methods used in this report are the Software Process
Assessment (SPA), CMM-Based Appraisal for both Internal Process Improvement (CBA IPI) and Standard CMMI® Appraisal Method for Process Improvement (SCAMPI). We do request and receive other Software CMM-based appraisals such as Software Capability Evaluations (SCE) and Interim Profiles. As our sampling size of these other methods increase, they will be reported here.
Company - Parent of the appraised entityA company can be a commercial or non-commercial firm, for-profit or not for-profit business, a research and development unit, a higher education unit, a government agency, or branch of service, etc.
Organization – a.k.a. Appraised entity The organization unit to which the appraisal results apply. An appraised entity can be the entire company, a selected business unit, units supporting a particular product line or service, etc.
Non-USA - Appraised entity whose geographic location is not within the United States. The parent of the appraised entity may or may not be based within the United States.
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 30© 2004 by Carnegie Mellon University
Report your Appraisal Results to the SEIThe briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI
In order to provide this information and service in the future, it will depend on this continued cooperation
Please visit:
http://www.sei.cmu.edu/sema/report.html
for forms, information, and instructions on how to report appraisals to the SEI
Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results
March 2004 Process Maturity Profile - Page 31© 2004 by Carnegie Mellon University
Contact InformationPlease visit:
http://www.sei.cmu.edu/sema/profile_about.html
and review the information provided before contacting:
SEI Customer Relations (412) 268-5800SEI FAX number (412) 268-5758
Internet [email protected]
Mailing AddressCustomer RelationsSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890