Date post: | 22-Dec-2015 |
Category: |
Documents |
Upload: | jocelyn-harper |
View: | 222 times |
Download: | 0 times |
SERA 2003Keynote AddressResearch Review
San Francisco, CAJune 25, 2003
A forthcoming Addison Wesley Book
©USC-CSE6/25/03 1
6/25/03 ©USC-CSE 2
Background
• Two approaches to software development – Disciplined (SW-CMM, document-based, heavy
process)– Agile (XP, tacit knowledge, light process)
• Both have strengths and weaknesses• Agile and disciplined proponents are
believers• Many of us are perplexed
6/25/03 ©USC-CSE 3
The Agile Manifesto - I
• Individuals and interactions over processes and tools• Working software over comprehensive documentation• Customer collaboration over contract negotiation• Responding to change over following a plan
We are uncovering better ways of developing software by doing it and helping others do it.Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more.
6/25/03 ©USC-CSE 4
The Agile Manifesto – II
• Our highest priority is to satisfy the customerthrough early and continuous deliveryof valuable software.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Business people and developers must work together daily throughout the project.
6/25/03 ©USC-CSE 5
The Agile Manifesto – III
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
6/25/03 ©USC-CSE 6
The Agile Manifesto – IV
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity – the art of maximizing the amount of work not done – is essential.
• The best architectures, requirements, and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
6/25/03 ©USC-CSE 7
Various Agile Methods Available
• Adaptive Software Development (ASD)• Agile Modeling• Crystal methods• Dynamic System Development Methodology
(DSDM)* eXtreme Programming (XP)• Feature Driven Development• Lean Development• Scrum
6/25/03 ©USC-CSE 8
XP Principles – I
• Philosophy: Take known good practices and push them to extremes
• “If code reviews are good, we’ll review code all the time”
• “If testing is good, we’ll test all the time”
• “If design is good, we’ll make it part of everybody’s daily business”
6/25/03 ©USC-CSE 9
XP Principles – II
• “If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality”
• “If architecture is important, everybody will work defining and refining the architecture all the time”
• “If integration testing is important, then we’ll integrate and test several times a day”
6/25/03 ©USC-CSE 10
XP Principles – III
• “If short iterations are good, we’ll make the iterations really, really short – seconds and minutes and hours, not weeks and months and years”
• “If customer involvement is good, we’ll make them full-time participants”
6/25/03 ©USC-CSE 11
XP: The 12 Practices
• The Planning Game• Small Releases• Metaphor• Simple Design• Testing• Refactoring
• Pair Programming• Collective Ownership• Continuous Integration• 40-hour Week• On-site Customer• Coding Standards
-Used generatively, not imperatively
6/25/03 ©USC-CSE 12
Counterpoint: A Skeptical View – ILetter to Computer, Steven Rakitin, Dec.
2000
“individuals and interactions over processes and tools” Translation: Talking to people gives us the flexibility to do whatever we
want in whatever way we want to do it. Of course, it’s understood that we know what you want - even if you don't.
“working software over comprehensive documentation” Translation: We want to spend all our time coding. Real programmers
don’t write documentation.
6/25/03 ©USC-CSE 13
Counterpoint: A Skeptical View – IILetter to Computer, Steven Rakitin, Dec.
2000
“customer collaboration over contract negotiation” Translation: Let's not spend time haggling over the details, it only
interferes with our ability to spend all our time coding. We’ll work out the kinks once we deliver something...
“responding to change over following a plan” Translation: Following a plan implies we would have to spend time
thinking about the problem and how we might actually solve it. Why would we want to do that when we could be coding?
6/25/03 ©USC-CSE 14
Sources of Perplexity
• Distinguishing method use from method misuse– Claiming XP use when simply “not documenting”– “CMM Level 4 Memorial Library” of 99 2-inch binders
• Overgeneralization based on the most visible instances– XP is Agile; CMM is Discipline
• Multiple definitions– Quality: customer satisfaction or compliance?
• Claims of universality– The pace of IT change is accelerating and Agile
methods adapt to change better than disciplined methods therefore Agile methods will take over the IT world.
– Software development is uncertain and the SW-CMM improves predictability therefore all software developers should use the SW-CMM
6/25/03 ©USC-CSE 15
Sources of Perplexity (2)
• Early success stories– Chrysler project that successfully birthed XP was
later cancelled– Cleanroom has never made it into the mainstream
• Purist interpretations (and internal disagreements)– “Don't start by incrementally adopting parts of
XP. Its pieces fit together like a fine Swiss watch“– "An advantage of agile methods is that you can
apply them selectively and generatively."– "If you aren't 100% compliant with SW CMM Level
3, don't bother to bid" – "With the CMMI continuous interpretation, you can
improve your processes in any order you feel is best."
6/25/03 ©USC-CSE 16
Key Definitions
• Agile method – – one which fully adopts the four value propositions
and twelve principles in the Agile Manifesto.• Discipline – (per Webster)
– control gained by enforcing obedience or order; – orderly or prescribed conduct or pattern of
behavior; – self-control.
• Plan-driven – – a description for disciplined methods (order is
often defined in plans) • Plan – (per Webster)
– a method for achieving an end (a process plan); – an orderly arrangements of parts of an overall
design (a product plan). – We assume that such plans are documented.
6/25/03 ©USC-CSE 17
Finding Middle Ground
• A pragmatic view• Both approaches have “home grounds”• A broad range of environments and needs• Trends require better handling of rapid
change• Processes should be the right weight for the
job• We believe risk-based analysis is the key to
finding middle ground
6/25/03 ©USC-CSE 18
Management Characteristics
• Customer Relations– A: Dedicated, collocated– D: Contractual– A: Trust through working software and
participation– D: Trust through process maturity evaluations
• Planning and Control– A: Means to an end– D: Anchor processes, communication
• Project Communications– A: Tacit, interpersonal knowledge– D: Explicit, documented knowledge
6/25/03 ©USC-CSE 19
Technical Characteristics
• Requirements– A: Adjustable, informal stories– D: Formally baselined, complete, consistent
specifications
• Development– A: Simple Design– D: Architecture-based design
• Testing– A: Automated, test-driven– D: Planned, requirements-driven
6/25/03 ©USC-CSE 20
Technical Characteristics (2)Cost of Change
Cos
t of
Cha
nge
Time
Beck
LiLi
6/25/03 ©USC-CSE 21
Technical Characteristics (3)How Much Architecting?
Beck
LiuLiu
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60
Percent of Time Added for Architecture and Risk Resolution
Per
cen
t of T
ime
Ad
ded
to O
vera
ll S
ched
ule
Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution
Added Schedule Devoted to Rework(COCOMO II RESL factor)
Total % Added Schedule
Sweet Spot Drivers:
Rapid Change: leftward
High Assurance: rightward
Sweet spot
10 KSLOC
100 KSLOC
10,000 KSLOC
6/25/03 ©USC-CSE 22
Personnel Characteristics
• Customers– A: CRACK customers throughout development– D: CRACK customers early
• CRACK: Collaborative, Representative, Authorized, Committed, and Knowledgeable
• Developers– A: Heavy mix of high caliber throughout– D: Heavy mix early with lower capability later
• Culture– A: Many degrees of freedom (Thrives on
chaos)– D: Clear policies and procedures (Thrives on
order)
6/25/03 ©USC-CSE 23
Personnel Characteristics (2)Modified Cockburn Levels
(Ski
ll, U
nders
tandin
g)
(Formality, Documentation)
(Ski
ll, U
nders
tandin
g)
(Formality, Documentation)
Level Characteristics
3 Able to revise a method (break its rules) to fit an unprecedented new situation
2 Able to tailor a method to fit a precedented new situation
1A With training, able to perform discretionary method steps (e.g., sizing stories to fit increments, composing patterns, compound refactoring, complex COTS integration). With experience can become Level 2.
1B With training, able to perform procedural method steps (e.g. coding a simple method, simple refactoring, following coding standards and CM procedures, running tests). With experience can master some Level 1A skills.
-1 May have technical skills, but unable or unwilling to collaborate or follow shared methods.
6/25/03 ©USC-CSE 24
Summary of Home GroundsCharacteristics Agile Disciplined
Application
Primary Goals Rapid value; responding to change Predictability, stability, high assurance
Size Smaller teams and projects Larger teams and projects
Environment Turbulent; high change; project-focused Stable; low-change; project/organization focused
Management
Customer Relations
Dedicated on-site customers; focused on prioritized increments
As-needed customer interactions; focused on contract provisions
Planning/Control
Internalized plans; qualitative control Documented plans, quantitative control
Communications
Tacit interpersonal knowledge Explicit documented knowledge
Technical
Requirements Prioritized informal stories and test cases; undergoing unforseeable change
Formalized project, capability, interface, quality, forseeable evolution requirements
Development Simple design; short increment; refactoring assumed inexpensive
Extensive design; longer increments; refactoring assumed expensive
Test Executable test cases define requirements, testing
Documented test plans and procedures
Personnel
Customers Dedicated, collocated CRACK* performers CRACK* performers, not always collocated
Developers At least 30% full-time Cockburn level 2 and 3 experts; no Level 1B or -1 personnel**
50% Cockburn Level 3s early; 10% throughout; 30% Level 1B’s workable; no Level -1s**
Culture Comfort and empowerment via many degrees of freedom (thriving on chaos)
Comfort and empowerment via framework of policies and procedures (thriving on order)
* Collaborative, Representative, Authorized, Committed, Knowledgeable** These numbers will particularly vary with the complexity of the application
6/25/03 ©USC-CSE 25
Five Critical Decision Factors
• Represent five dimensions• Size, Criticality, Dynamism, Personnel,
Culture Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Agile
Disciplined
6/25/03 ©USC-CSE 26
Two Case Studies
• Show how that agile and disciplined methods can be extended
• Provide examples of success and lessons learned
• ThoughtWorks Lease Management– Agile extended with disciplined
• CCPDS-R– Disciplined overlaid with agile concepts
6/25/03 ©USC-CSE 27
Thoughtworks Lease Management
• XP replaced ineffective traditional development• Problems when project moved beyond XP
assumptions– The effort to develop or modify a story really does not
increase with time and story number– Trusting people to get everything done on time is
compatible with fixed schedules and diseconomies of scale
– Simple design and YAGNI scale up easily to large projects
• Disciplined practices enabled XP to scale up– High-level architectural plans to provide essential
big-picture information– More careful definition of milestone completion
criteria to avoid “finishing” but not finishing– Use of design patterns and architectural solutions
rather than simple design to handle foreseeable change
6/25/03 ©USC-CSE 28
CCPDS-R
• USAF Command Center Processing and Display System Replacement for early missile warning
• Government contract environment required disciplined approach, project needed agility
• Practices implemented to provide agility mapped well to the Agile Manifesto – Individuals and Interactions over Processes and
Tools• Milestone content was redefined • Architecture was organized around the performers’ skill levels
– Working Software over Comprehensive Documentation
• Later PDR demonstrated working software for high-risk areas – Customer Collaboration Over Contract
Negotiation • Used COCOMO for cost and schedule negotiations
– Responding to Change Over Following a Plan • Automated plans• Architecture for re-use saved effort
6/25/03 ©USC-CSE 29
CCPDS-R (2) Cost of Change
RATIONALS o f t w a r e C o r p o r a t I o n
Project Development Schedule15 20 25 30 35 40
40
30
20
10
Design Changes
Implementation Changes
MaintenanceChanges and ECP’s
HoursChange
RATIONALS o f t w a r e C o r p o r a t I o n
Project Development Schedule15 20 25 30 35 40
40
30
20
10
Design Changes
Implementation Changes
MaintenanceChanges and ECP’s
HoursChangeHours
Change
Cos
t of
Cha
nge
Time
Beck
6/25/03 ©USC-CSE 30
Using Risk to Balance Discipline and Agility - Overview
Step 5. Execute and Monitor
Step 4. Tailor Life Cycle
Step 3. Architecture Analysis
Step 1. Risk Analysis
Step 2. Risk Comparison
Rate the project’s environmental, agility-oriented and discipline-
oriented risks.
Uncertain about
ratings?
Buy information via prototyping, data
collection and analysis
Compare the agile
and disciplined
risks
Go Risk-driven Agile
Agility risks dominate
Discipline risks dominate
Architect application to encapsulate agile parts
Go Risk-driven Agile in agile
parts; Go Risk-driven Disciplined
elsewhere
Yes
No
Go Risk-driven Disciplined
Tailor life cycle process around anchor point
commitment milestones
Monitor progress and risks/opportunities,
readjust balance and process as appropriate
Neither dominate
Deliver incremental capabilities according to
strategy
6/25/03 ©USC-CSE 31
Risks Used in Risk Comparison
• Environmental risks– Technology uncertainties– Many stakeholders– Complex system-of-systems
• Agility-oriented risks– Scalability– Use of simple design– Personnel turnover– Too-frequent releases– Not enough agile-capable people
• Discipline-oriented risks– Rapid change– Need for rapid results– Emergent requirements– Not enough discipline-capable people
6/25/03 ©USC-CSE 32
Three Examples
• Agent-based systems• Small – Event Managers application• Medium – SupplyChain.com application• Large – National Information System for
Crisis Management (NISCM) application• Each example results in a development
strategy based on risk analyses
6/25/03 ©USC-CSE 33
Event Managers Project
• Small startup company • Diverse set of smaller agent-based
planning applications • This project: support for managing the
infrastructure and operations of conferences and conventions
• Widening variety of options and interdependencies, makes an agent-based approach highly attractive
6/25/03 ©USC-CSE 34
Event Managers Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Risk Items Risk Ratings Event Managers
Environmental Risks E1. Technology uncertainties E2. Many stakeholders E3. Complex system of systems Risks of using Agile Methods A1. Scalability A2. Use of simple design A3. Personnel turnover A4. Too-frequent releases A5. Not enough agile-capable people Risks of using disciplined methods D1. Rapid change D2. Need for rapid results D3. Emergent requirements D4. Not enough discipline-capable people
Risk rating scale: - minimal risk - moderate risk
- Serious but manageable risk - Very serious but manageable risk - Show-stopper risk
6/25/03 ©USC-CSE 35
Event Managers Strategy
1. StartupTeambuilding and Shared
Vision
2. Design, Development and Deployment
• Establish CRACK joint management team
• Develop shared vision, results chain, life cycle strategy, infrastructure choices
• Establish commitment to proceed, incremental capabilities, backlog
• Develop next-increment features
• Test, exercise, deploy new increment• Re-prioritize next-increment features• Analyze lessons learned; prepare strategy for next increment
Customer Management; Representative Users
Joint Developer-Customer Agile Team
6/25/03 ©USC-CSE 36
SupplyChain.com Profile
• Turnkey agent-based supply chain management systems
• Distributed, multi-organization teams of about 50 people
• Parts of applications are relatively stable, while others are highly volatile
• Architectures are driven by a few key COTS packages that are also continually evolving
6/25/03 ©USC-CSE 37
SupplyChain.com Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Risk Items Risk Ratings SupplyChain.com
Environmental Risks E1. Technology uncertainties E2. Many stakeholders E3. Complex system of systems Risks of using Agile Methods A1. Scalability A2. Use of simple design A3. Personnel turnover A4. Too-frequent releases A5. Not enough agile-capable people Risks of using disciplined methods D1. Rapid change D2. Need for rapid results D3. Emergent requirements D4. Not enough discipline-capable people
Risk rating scale: - minimal risk - moderate risk
- Serious but manageable risk - Very serious but manageable risk - Show-stopper risk
6/25/03 ©USC-CSE 38
SupplyChain.com Strategy
Startup 1. Teambuilding and Shared
Vision
3. Design, Development and Deployment2. Systems Definition and Architecting
Stakeholders
Furnish CRACK representatives and alternates
Staff and organize to cover all success-critical risk areas
• Develop benefits-realization results chains
• Identify missing stakeholders
• Enhance mutual understanding
• Explore goals, options, technology, prototypes, risks
Review; Commit
to proceed
• Elaborate supply chain operational concept, prototypes, transition strategy
• Evaluate and determine COTS, reuse, and architecture choices
• Prioritize and sequence desired capabilities, outstanding risks
• Establish project organization, overall process, and feature teams
Review; Commit
to proceed
• Ensure representative exercise of incremental capabilities
• Monitor progress and new operational development
• Monitor project progress, risk resolution, and new technology developments
• Identify and work critical project-level actions
• Develop and integrate new feature increments
• Communicate proposed redirections
• Prepare for and execute acceptance tests, installation, operational evolution
• Analyze feedback on current version
• Reprioritize features
• Prepare strategy for next increment
• Consolidate process lessons learned
• Use risk to determine content of artifacts
Project Manager andRisk Management Team
Agile Feature Team
6/25/03 ©USC-CSE 39
NISCM Profile
• Broad coalition of government agencies and critical private-sector organizations
• Support cross-agency and public-private sector coordination of crisis management activities
• Adaptive mobile network, virtual collaboration, information assurance, and information integration technology
• Private-sector system-of-systems integration contractor
6/25/03 ©USC-CSE 40
Risk Items Risk Ratings NISCM
Environmental Risks E1. Technology uncertainties E2. Many stakeholders E3. Complex system of systems Risks of using Agile Methods A1. Scalability — A2. Use of simple design — A3. Personnel turnover A4. Too-frequent releases A5. Not enough agile-capable people — Risks of using disciplined methods D1. Rapid change D2. Need for rapid results D3. Emergent requirements D4. Not enough discipline-capable people
Risk rating scale: - minimal risk - moderate risk
- Serious but manageable risk - Very serious but manageable risk - Show-stopper risk
NISCM Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
6/25/03 ©USC-CSE 41
Startup Teambuilding and Shared
Vision
Design, Development and Deployment
Systems Definition and Architecting
Stable, Higher-criticality Module Teams
Stakeholders
Furnish CRACK representatives and alternates
• Staff and organize to cover all success-critical risk areas
• Prepare for and select competitors for concept definition
• Develop results chains• Identify missing
stakeholders• Enhance understanding• Explore goals, options,
technology, prototypes, risks
• Negotiate mutually satisfactory objectives and priorities
• Formulate top-level operational concept, requirements, architecture, life cycle plan, feasibility rational
• Select winning system integration contract
Hold Life Cycle
Objective Review. If feasibility
rationale is valid,
commit to proceed
• Prepare for/select competitors for component subcontract definition
• Elaborate operational concept, prototypes, transition strategy
• Evaluate/determine COTS, reuse, and architecture choices
• Develop/ validate critical-infrastructure software
• Prioritize/sequence desired capabilities, outstanding risks
• Formulate definitive operational concept, requirements, architecture, life cycle plan, feasibility rationale
• Use to solicit/select component subcontractors
Hold Life Cycle
Architecture Review. If feasibility
rationale is valid,
commit to proceed
• Ensure representative exercise of incremental capabilities
• Monitor progress and new operational development
• Monitor project progress, risk resolution, and new technology developments
• Identify/work critical project-level actions
• Continuously integrate/ test growing software infrastructure and components
Dynamic, Lower-criticality Module Teams
• Develop compatible component-level prototypes, requirements, architectures, plans, critical software, feasibility rationale
• Classify modules by likely stability and criticality
Organize into disciplined and agile module teams
Plan, specify, develop stable, higher-criticality module increments
Plan, specify, develop dynamic, lower-criticality module increments
• Communicate proposed redirections
• Prepare for and execute acceptance tests, installation, operational evolution
• Analyze feedback on current version
• Reprioritize features
• Prepare strategy for next increment
• Use risk to determine content of artifacts
NSO and I3-led ProjectRisk Management Team
NISCM Strategy
6/25/03 ©USC-CSE 42
Risk Profiles Across Examples
Risk Items Risk Ratings Event Managers SupplyChain.com NISCM
Environmental Risks E1. Technology uncertainties E2. Many stakeholders E3. Complex system of systems Risks of using Agile Methods A1. Scalability — A2. Use of simple design — A3. Personnel turnover A4. Too-frequent releases A5. Not enough agile-capable people — Risks of using disciplined methods D1. Rapid change D2. Need for rapid results D3. Emergent requirements D4. Not enough discipline-capable people Risk rating scale: - minimal risk - moderate risk
- Serious but manageable risk - Very serious but manageable risk - Show-stopper risk
42©USC-CSE6/10/03
6/25/03 ©USC-CSE 43
Reflections on the Agile Manifesto
• The four value preferences are not exclusive-or’s – Processes and tools can empower individuals and
interactions– Documentation can help people understand complex
software – Contract negotiation can be an integral part of
customer collaboration– When changes are foreseeable, following a plan can
be the best way to respond to change
• Each pair of alternatives can reinforce each other – Can use risk to help find best balance
• People and their value propositions are top priority– Not just developers and customers, but end users,
maintainers, interoperators, general public, …
6/25/03 ©USC-CSE 44
Hybrid Agile/Plan-Driven Methods
• Early agility to determine emergent requirements– Agile user-interface prototyping
• Early planning to avoid surprises, point solutions– Plans for test data, drivers, oracles, tools, facilities– Architectures for assuring scalability, interoperability
• Early hybrid approach for complex COTS selection– Planned evaluation criteria, approach, priorities– Agile evaluations and iteration of approach
• Main development tailored to agile, plan-driven home grounds– Architect to modularize around sources of rapid change– Apply agile methods to rapidly changing lower-criticality
modules– Planned approach to multi-site relativity stable, higher-
criticality modules
6/25/03 ©USC-CSE 45
Large System Life Cycle Planning/Agility Framework
KPP’s:cost, schedule, performance, dependability, interoperability, …
Desired
FOC
DesiredIOC
Acceptable IOC
Desired FOC
Desired
Prioritized Capabilities
IOC Trade Space
Full Operational Capability(FOC) Architecture Support
AcceptableInitial Operational Capability (IOC)
Key
Per
form
ance
Par
amet
ers
(KP
P’s
)
6/25/03 ©USC-CSE 46
Conclusions
• Neither agile nor disciplined methods provide a silver bullet
• Agile and disciplined methods have home grounds where one clearly dominates the other
• Future trends are toward application developments that need both agility and discipline
• Some balanced methods are emerging• It is better to build your method up than to
tailor it down• Methods are important, but potential silver
bullets are more likely to be found in areas dealing with people, values, communications, and expectations management.