/2012

254

12/09/

Quality AssuranceSoftware Development Processes

SO

FTE

NG

Software Development Processes

eala

nd Part II - Lecture 2

klan

d | N

ew Z

eve

rsity

of A

uck

The

Uni

v

1

The Standish “Chaos” R p rts

/2012

ReportsReports on statistics about IT

254

12/09/

pprojects (data for 2009)• 32% of all projects succeeded

SO

FTE

NG

(delivered on time, on budget, with required features and functions)

eala

nd

functions) • 44% are challenged (late, over

budget and/or with less than the

klan

d | N

ew Z

e

required features and functions)• 24% have failed (cancelled prior

to completion or delivered and

Among the suspected causes:

i d

vers

ity o

f Auc

k to completion or delivered and never used)

poor estimates and poor planning

The

Uni

v

http://blog.standishgroup.com/ 2

Today’s Outline/2012

y 2

5412/09/

• Software Development ProcessesWh t i it?

SO

FTE

NG

– What is it?– Phases of a process

W f ll d l

eala

nd

– Waterfall model• Capability Maturity Model (CMM)

klan

d | N

ew Z

eve

rsity

of A

uck

The

Uni

v

3

/2012

254

12/09/

SO

FTE

NG

Software Development P

eala

nd

Processes

klan

d | N

ew Z

e

He who fails to plan,

vers

ity o

f Auc

k pplans to fail(Proverb)

The

Uni

v

4

Software Development Pr c ss

/2012

ProcessGeneric plan for a software project

254

12/09/

p p j

1. What has to be done? (-> tasks/activities/steps)time

SO

FTE

NG

2. Why do a task? (-> outcomes, produced artifacts)3. When should it be done? (-> schedule)4 Wh d i ( l l ibili i )

eala

nd

4. Who does it? (-> people, roles, responsibilities)5. How should it be done? (-> methods, standards, tools)

klan

d | N

ew Z

e

• Many different processes exist• No single process suitable for every project

vers

ity o

f Auc

k • No single process suitable for every project (no “one size fits all”)

• Using a process can improve the quality of the product

The

Uni

v

5

s ng a proc ss can mpro th qua ty of th pro uct

Phases of a Process 1/2012 Simple old process model:

254

12/09/

p p1. Write/change program (implementation phase)2. Find defects, go back to 1.

SO

FTE

NG

Problem:– Ad hoc changes over time mess up program

eala

nd

– Ad hoc changes over time mess up program structure!!!

– Result: further changes cost more and more.

klan

d | N

ew Z

e g

Solution:A d i h i hi h ll t t

vers

ity o

f Auc

k – A design phase in which overall program structure is defined

– Changes of the implementation are allowed but

The

Uni

v

6

Changes of the implementation are allowed, but have to follow the design

Phases of a Process 2/2012 Improved process model:

254

12/09/

p p1. Define a design for the program (design phase)2. Write/change program (implementation phase)

SO

FTE

NG

3. Find defects, go back to 2.

Problem:

eala

nd

Problem:– Does the program do what the user wants?– Result: program may be rejected by the user

klan

d | N

ew Z

e Result: program may be rejected by the user.

Solution:

vers

ity o

f Auc

k

– An requirements phase in which the requirements for the program are specifiedThe design is defined so that the user’s

The

Uni

v

7

– The design is defined so that the user s requirements are satisfied

Phases of a Process 3/2012

Even better process model:

254

12/09/

p1. Analyze the requirements of the user

(requirements phase)

SO

FTE

NG

2. Define a design for the program (design phase)3. Write/change program (implementation phase)

eala

nd

g p g p p4. Find defects, go back to 3.

klan

d | N

ew Z

e

You learned in the first half of the QA course:• Testing needs to be planned and prepared

vers

ity o

f Auc

k g p p p• Step 4 is important and should have its own phase• In the testing phase, the product is systematically

The

Uni

v

8

n th t st ng phas , th pro uct s syst mat ca y checked for defects

Software Development Lif c cl (SDLC)

/2012

Lifecycle (SDLC)Phase Result

254

12/09/

Requirements Specification of the user requirementsDesign Specification of the overall program

t t

SO

FTE

NG

structureImplementation Executable program (“alpha” quality)T i E bl (“ l ” li )

eala

nd

Testing Executable program (“release” quality)

• These are just the most common phases!!!

klan

d | N

ew Z

e j m mm p• Other phases may include:

deployment, operation, support, training, maintenance

vers

ity o

f Auc

k p y p pp g• All the phases together are sometimes referred to as

Software Development Life Cycle (SDLC)

The

Uni

v

9

Waterfall Model/2012

• Old fashioned model of how a process should work

254

12/09/

p• Go through the phases one after another• A.k.a. stagewise model, linear sequential model

SO

FTE

NG

g m , q m

Requirements Design Implementation Testing

eala

nd Problem:Oft h s i tif ts f i s h s s

klan

d | N

ew Z

e – Often changes in artifacts of previous phases necessaryIn big systems many requirements become visible

vers

ity o

f Auc

k – In big systems many requirements become visible only after analysis phase or change over time

– Design flaws are often discovered during

The

Uni

v

10

Design flaws are often discovered during implementation and testing

Waterfall Model with Backflow

/2012

with Backflow• Extends the stagewise model with possibility to go

254

12/09/

g p y gback to previous phase (“backflow”)

• Defects from the previous phase can still be corrected

SO

FTE

NG

corrected

R i D i

eala

nd

Requirements Design Implementation Testing

klan

d | N

ew Z

e

• Step towards iterative / incremental development(heavily used in most modern processes)

vers

ity o

f Auc

k

• Still very simplistic• In reality activities of different phases often happen

t th ti

The

Uni

v

11at the same time

/2012

254

12/09/

SO

FTE

NG

The Capability Maturity M d l (CMM)

eala

nd

Model (CMM)

klan

d | N

ew Z

eve

rsity

of A

uck

Never eat more than you can lift(Miss Piggy)

The

Uni

v

12

Capability Maturity Model (CMM)

/2012

(CMM)How can we guarantee high-quality results?

254

12/09/

g g q y

Idea of CMM: h h l ld h h l d

SO

FTE

NG

• A high-quality process yields a high-quality product• Let’s make our process high-quality!

eala

nd

CMM• Describes "the key elements of an effective process"

klan

d | N

ew Z

e D r y m n f n ff pr– Evolution from ‘immature’ process to ‘mature,

disciplined’ process

vers

ity o

f Auc

k

– Key practices for meeting goals for cost, schedule, functionality, and product quality

• Can be used to measure and improve the maturity of a

The

Uni

v

13

• Can be used to measure and improve the maturity of a process

Capability Maturity Model (CMM)

/2012

(CMM)• Developed by the U.S. Department of Defence

254

12/09/

p y pSoftware Engineering Institute (SEI) in the 1980s

• Has been continuously revised since thenM ti ti bj ti l ti f t t f

SO

FTE

NG

• Motivation: objective evaluation of contractors for military software projects

• Used for many government projects

eala

nd

Used for many government projects• Categorizes software development organizations into

one of five process maturity levels

klan

d | N

ew Z

e p y• Each maturity level defines:

– A certain capability of producing quality software

vers

ity o

f Auc

k

– Key process areas (what is done?)– Key practices (how is it done?)

The

Uni

v

14• Level 1 worst, level 5 best

CMM Level 1: Initial/2012 • No stable environment for developing and

254

12/09/

p gmaintaining software

• Constant changes of the process make it unpredictable ("ad hoc")

SO

FTE

NG

unpredictable ( ad hoc ).– Umpredictable cost, schedule,

functionality, and product quality

eala

nd

functionality, and product quality– Performance depends on individuals

and varies with their innate skills, k l d d ti ti ("h i ")

klan

d | N

ew Z

e knowledge, and motivations ("heroic")– Irregular work schedule:

long hours and deadline stress

vers

ity o

f Auc

k long hours and deadline stress• Most organizations are only level 1

The

Uni

v

15

CMM Level 2: Managed/2012g

• Effective management processes for

254

12/09/

g psoftware projects are institutionalized– Project planning

Pl

SO

FTE

NG

– Project tracking and oversight– Requirements management

Plan

eala

nd

q g– Configuration management (CM)

• Planning and managing new projects is

klan

d | N

ew Z

e g g g p jbased on experience with similar projects

• Successful practices from earlier projects

vers

ity o

f Auc

k

can be repeated

The

Uni

v

16

CMM Level 3: Defined/2012 • Process is standardized and documented by SE process

group (SEPG)

254

12/09/ group (SEPG)

– Process definition, process focus• Standard process can be tailored to the unique

SO

FTE

NG

• Standard process can be tailored to the unique characteristics of a project

• Effective SE practices used

eala

nd

– Software product engineering– Intergroup coordination

klan

d | N

ew Z

e

– Peer reviews• Training program

so that everybody can fulfill their roles

vers

ity o

f Auc

k so that everybody can fulfill their roles

The

Uni

v

17

CMM Level 4:Qu ntit tiv l M n d

/2012

Quantitatively ManagedQuantitative process management

254

12/09/ • The process is instrumented with well-defined

measurements in organizational measurement program• Productivity and quality are measured for all

SO

FTE

NG

• Productivity and quality are measured for all important activities

• Organization-wide database for collecting and

eala

nd

analyzing the surveyed data • Quantitative quality goals for both software

products and processes

klan

d | N

ew Z

e products and processesSoftware quality management:• Process is controlled to operate within acceptable limits

vers

ity o

f Auc

k

• Predictably high quality

The

Uni

v

18

CMM Level 5: Optimizing/2012

p g

• Continuous process improvement

254

12/09/ • Use surveyed data to analyse cost benefit

of new technologies and process changes– Technology change management

SO

FTE

NG

– Technology change management– Process change management

• Identify and disseminate innovation

eala

nd

y• Defect prevention

– Analyze causes of defects

klan

d | N

ew Z

e

– Prevent known types of defects from recurring– Indentify weaknesses and improve proactively

• Disseminate experience between projects

vers

ity o

f Auc

k • Disseminate experience between projects

The

Uni

v

19

CMM Summary/2012

yLevel Characteristics Key process areas1 l U bl N

254

12/09/ 1 Initial Unstable environment

Unpredictable, ad hocNone

2 Managed Management processes Requirements management

SO

FTE

NG

Plan

2 Managed Management processesBased on experience

Requirements management, project planning, tracking and oversight, CM

eala

nd

3 Defined Standardized, docu-mented processEffective SE practices

Process definition & focus, training program, SE, peer review

klan

d | N

ew Z

e p , p4 Quant.

ManagedMeasurement programPredictably high quality

Quantitative process management, software quality management

vers

ity o

f Auc

k quality management5 Optimizing Process improvement

Analyze defectsTechnology & process change management,

The

Uni

v

20

yDisseminate experience

g g ,defect prevention

CMM Criticism/2012

• CMM has failed to take over the world;

254

12/09/ CMM has failed to take over the world;

most organizations still level 1• Commercially more successful methodologies, e.g. RUP• CMM is not a recipe or guarantee for success; it may just

SO

FTE

NG

• CMM is not a recipe or guarantee for success; it may just increase its probability

• Little validation of the cost savings below 4th level

eala

nd

• Too much bureaucratic overhead– Well suited for bureaucratic organizations,

e.g. government agencies, large corporations

klan

d | N

ew Z

e g g g g p– Focus on "perfectly completed forms" rather than application

development, client needs or the market– Process may impede meeting schedule in cases where time to

vers

ity o

f Auc

k Process may impede meeting schedule in cases where time to market with some product is more important than quality and functionality

• Promoting process over substance

The

Uni

v

21

Promoting process over substance (e.g. predictability over service provided to end users)

Other Quality Frameworks/2012

Q y

SPICE (Software Process Improvement & Capability dEtermination)ISO t CMM

254

12/09/ • ISO answer to CMM

• Basically the 5 levels of CMM plus level 0 for incomplete/nonexistent process

SO

FTE

NG

ISO9000• Family of ISO standards for quality management systems

eala

nd

• Certification of compliance possible, only pass or fail• 8 quality management principles

1. Focus on your customers

klan

d | N

ew Z

e y2. Provide leadership 3. Involve your people4 Use a process approach

vers

ity o

f Auc

k 4. Use a process approach 5. Take a systems approach6. Encourage continual improvement 7 Get the facts before you decide

The

Uni

v

22

7. Get the facts before you decide 8. Work with your suppliers

Today’s Summary/2012

y y

• Software development processes define a

254

12/09/

p pgeneric plan for a software project– Processes are often divided into phases

SO

FTE

NG

– Often iterations of phases are needed• The quality of processes can be rated by

eala

nd

q y p ymaturity models like CMM– CMM defines hierarchy of 5 levels:

klan

d | N

ew Z

e

initial, managed, defined,quantitatively managed, optimizingM t it d l h l t i

vers

ity o

f Auc

k – Maturity models can help to improve a processReference:

Software Engineering Institute Capability Maturity Model for

The

Uni

v

23

Software Engineering Institute. Capability Maturity Model for Software, Version 1.1. Technical Report. 1993. http://www.sei.cmu.edu/reports/93tr024.pdf

Quiz/2012

Q

1. What does a software development process define?

254

12/09/

p pName 5 aspects that are defined.

SO

FTE

NG

2. What are the 4 most common phases of a software development process?

eala

nd 3. What are the key characteristics of each of the 5 CMM maturity levels?

klan

d | N

ew Z

e CMM maturity levels?

vers

ity o

f Auc

kTh

e U

niv

24