Finding Discipline in an Agile Acquisition ProcessAgile Acquisition Process
Tricia OberndorfMary Ann LaphamMichael BandorCharles “Bud” Hammons
Software Engineering InstituteSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213
18 May 2011
© 2011 Carnegie Mellon University
18 May 2011
Report Documentation Page Form ApprovedOMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.
1. REPORT DATE 18 MAY 2011 2. REPORT TYPE
3. DATES COVERED 00-00-2011 to 00-00-2011
4. TITLE AND SUBTITLE Finding Discipline in an Agile Acquisition Process
5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Carnegie Mellon University,Software Engineering Institute,Pittsburgh,PA,15213
8. PERFORMING ORGANIZATIONREPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES Presented at the 23rd Systems and Software Technology Conference (SSTC), 16-19 May 2011, Salt LakeCity, UT. Sponsored in part by the USAF. U.S. Government or Federal Rights License
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as
Report (SAR)
18. NUMBEROF PAGES
24
19a. NAME OFRESPONSIBLE PERSON
a. REPORT unclassified
b. ABSTRACT unclassified
c. THIS PAGE unclassified
Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
Outline
The QuestionOn “Rigor”On RigorA New IT Acquisition ProcessDiscipline in the Existing Processp gDiscipline in the New IT Acquisition ProcessRecommendations
2Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
The Question:
How can rigor be accomplished within DoD’s new IT Acquisition Process?q• In particular: how can the new IT Acquisition Process maintain rigor
similar to that found in today’s traditional approach while still achieving the objectives of a more flexible, responsive process?g j p p
3Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Rigor – What Do We Really Want?
Rigor: 1a (1) : harsh inflexibility in opinion, temper, or judgment : severity
(2) : the quality of being unyielding or inflexible : strictness…
b : an act or instance of strictness, severity, or cruelty
2 : a condition that makes life difficult, challenging, or uncomfortable;
3 : strict precision : exactness <logical rigor>
4a obsolete : rigidity, stiffnessb : rigidness or torpor of organs or tissue that prevents response to
stimuli c : rigor mortis
4Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Discipline, not Rigor
Discipline:1 : punishment
2 : a field of study
3 : training that corrects, molds, or perfects the mental faculties or moral character
4 : a rule or system of rules governing conduct or activity
5 a : control gained by enforcing obedience or orderg y gb : orderly or prescribed conduct or pattern of behaviorc : self-control
5Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Defense Acquisition Business Process
User Needs
Materiel IOCBA
Technology Engineering & Manufacturing Production & Operations &
C FOC
Technology Opportunities & Resources
(ProgramInitiation)
Solution Analysis
Technology Development Development Production &
Deployment
Systems Acquisition
pSupport
Sustainment
FRP DecisionReviewLRIP/IOT&E
Post CDRAssessment
Pre-Systems Acquisition
Materiel Development
Decision
PDR CDR
6Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Observations about Today’s Process
Frequent underlying problems in programs using this model include• lengthy gestation periods• management of requirements • failures in acceptance tests
Significant duration of typical program leads to heavy dependence on documentation to maintain “corporate memory.”
The undesirable side effects of early decisions, both technical and non-technical, only become visible years later,
usually during integration and test
7Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
DSB Report: New Acquisition Process for IT
8Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Materiel 0 Development Decision
Business Case Analysis and Development
lCD
Milestone Build Decision
! COD RELEASE 1
Architectural Development Development & Demonstration and Risk Reduction
Prototypes lteration1 I Iteration 2 I Iteration "N'
.., ___ Coordinated DOD stakeholder involvement .: Integrated DT I OT __ ..,.
+------ Upto2years -------~~..,..<~-- 6to18months
0 Development & Demonstration
Operations and
Support
Operations RELEASE 2 Prototypes
Iteration 1jneradon 2j1tenUon 3 and Support
lCD Initial Capability Document
COD Capabilities Development Document
0 Decision Point RELEASE "N"
Development &
Prototypes Demonstration
ltfration 1 !Iteration 21tteration 3
Continuous Technology/Requirements Development & Maturation
~ Software Engineering Institute I CarnegieMellon
Operations and Support
Tenets of a New IT Acquisition Process
Some key features of the new IT acquisition process:• frequent, usable releases of capability
– early, successive prototyping to support an evolutionary approach– deliver early and often– incremental and iterative development and testingincremental and iterative development and testing– executable and testable product
• early and continual involvement of the userti li d i t• rationalized requirements
• modular, open systems approach with standard interfaces• knowledgeable and experienced IT workforce• flexible, tailored processes
9Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Discipline in Today’s Approach
10Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
-......... ·~ -----
Integrated Defense Acquisition, Technology, and Logistics Life Cycle Management System
• ScJftware Engineering Institute I CarnegieMcllon
Features of Today’s Discipline
External scrutiny by decision makers• mandated decision events (Milestones A, B, C, ...)
Operational expectations documented in the Initial Capabilities Document (ICD) and Capabilities Development Document (CDD) artifacts• informal English language specifications
Numerous plans to document both business and technical approaches• by program offices and contractorsby program offices and contractors• from management of technology to deployment
Documentation of processes with compliance auditsi th t f ll d• ensuring that processes are followed
Financial performance reported against plan (earned value)Identification and management of risks
11Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Key Elements of Today’s Process
Requirements: key artifacts used to• govern developmentg p• form the basis of major reviews• orchestrate product evaluation, user acceptance, sell-off
Systems engineering documentation:Systems engineering documentation: • Subject Matter Experts (SMEs) at various levels
– act in part as advocates for their perception of user expectations( ) f• users sporadically involved (e.g., attend reviews) until field trials and
acceptance testingReviews:• progressively more detailed evaluations of information about product(s)• synchronized with major decision points to provide basis for decision
makers to appropriately intervene to influence development.
12Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Model of As-Is DisciplineAcquisition Side Operational/Execution Side
CONOPS,Acq Strategy,
Oversight (PMO & Above)
Oversight (COCOMS, Base Cmdrs & Above)
Risks ADMsICD, CDD
Doctrine,Tactics, “KPPs”
Info Support Plan
T h
Risks,Progress,Deltas[EVMS]
ADMs,Funding,Schedule,Cost, ..
Oversight/Insight
Tech Management
(PMO Chief Engrs & Below)
Materiel Mgmt (Logistics
Mgt/Brigade & Below)
TRD, Execution needs (log, training)
Plans,Risks,Progress,Deltas[‘prototype’
Plans,R
equireme
Development (IPT leads, contractors,
testers)
Users (Maintainers, Users)
[ prototype results]
Execution needs (maintenance & use)
nts
13Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
testers) (maintenance & use)
Discipline in the New IT Acquisition ProcessAcquisition Side Operational/Execution Side
Acq Strategy, CONOPS,
Oversight (PMO & Above)
Info Support Plan
ICD, CDD
Doctrine,Tactics, “KPPs”
Risks ADMs
Oversight (COCOMS, Base Cmdrs & Above)
T h
Risks,Progress,Deltas[EVMS]
ADMs,Funding,Schedule,Cost, ..
Tech Management
(PMO Chief Engrs & Below, Users)
Execution needs (log, training)
Materiel Mgmt (Logistics
Mgt/Brigade & Below)
Plans,Risks,Progress,Demos,Releases
Plans,O
peration aA
rchitectu
Development (IPT leads, Iteration
Teams Users)Execution needs (maintenance & use)
Releases[prototype results]
al ure
Users (Maintainers, End Users)
14Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Teams, Users) (maintenance & use)
Highlighted Differences
• The content of the information flows• Deltas include
– familiar items (deviations from plans and requirements)– use cases deferred to future iterations/releases, based on
experience in a given iteration/releasep g• Demonstrations and formal Releases provide feedback• Use cases:
take the place of functional requirements– take the place of functional requirements– give actionable specification of behavior as well as the context– provide direct mapping to testing and evaluation
The central role formerly served by requirements is replaced by the Operational Architecture.
15Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
The Operational Architecture
A structured representation of:• doctrine, tactics, and CONOPS
th t f th t f ll h t i b h i f th• the set of use cases that formally characterize behavior of the envisioned system in operational terms
• quality attributes that characterize performance and other system-level characteristics of the envisioned systemlevel characteristics of the envisioned system– beyond the functions the system will perform, e.g., security,
reliabilityth f t h l t b l d• the range of technology to be employed
• constraints such as mandated standardsEvolves • through the information and experience gained in each iteration• across multiple releases• becomes the living information about the system context
16Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
becomes the living information about the system context
Discipline in the New IT Acquisition Process1
External scrutiny by decision makers at mandated decision events as well as the end of iterations and releases• short duration of iterations and releases provides feedback to decision
makers on choices they personally made, enabling corrective actionsOperational expectations:• well-formed use cases more detailed than typical CDDs
– retains context and fine points influencing the behavior– more likely to be directly usable by development teammore likely to be directly usable by development team
• Operational Architecture more actionable explication of user expectations, constraints, quality attributes
Plans and compliance auditsPlans and compliance audits• frequent sprints of much shorter duration require less elaborate plans• compliance audits replaced by regular delivery of executable capability
17Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Discipline in the New IT Acquisition Process2
PLUSPersonnel• time-constrained iterations force personnel from all disciplines/roles to
work together repeatedly– amplifies experience in executing all parts of development cycle p p g p p y
together, from up-front systems analysis to test, integration, and deployment
Deltas• use case deferrals, shortfalls, test deficiencies are in domain-relevant
language of end users and decisions makers– avoids translation from technical to domain terminology gy
18Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Bottom Line
When we speak of discipline, we are advocating the creation of a more disciplined mechanism (structures + processes) to:• describe user expectations• enhance communications between user and acquisition/developer
communities• acknowledge there is of necessity an evolving understanding of what
is operationally required
The Operational Architecture is the key set of artifacts that document the results of the employment of this mechanism.
The processes and mechanisms establish the ongoing interaction p g gamong players in the user and acquiring organizations.
19Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Recommendations• Conduct effort to take this approach down to the next level of detail• Make some additions to the proposed process:
– Begin each iteration with an architecture segment• Assess architecture and potential extensions/revisions
– Begin each release cycle with a reassessment of the business case• Capture what has changed in system context and environmentCapture what has changed in system context and environment
• Revise the culture– Organizational structure, rewards systems, communication style,
decision-making style, staffing model (roles, team make-ups, etc.)decision making style, staffing model (roles, team make ups, etc.)• Look for personnel with special traits
– Self-starters, team players, multiple roles, communicators, adaptable• Institute new trainingInstitute new training
– Assists with culture change• Resolve issues in customer interaction
– Access to true end users is an essential element of the new process
20Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Access to true end users is an essential element of the new process
QUESTIONS ?QUESTIONS ?
21Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Contact Information
Mary Ann LaphamSenior Member of Tech Staff
U.S. MailSoftware Engineering Institute
Acquisition Support Program412-268-5498mlapham@sei cmu edu
g gCustomer Relations4500 Fifth AvenuePittsburgh, PA [email protected] USA
Tricia Oberndorf WebTricia OberndorfSenior Member of Tech StaffAcquisition Support Program412 973 3459
Webwww.sei.cmu.edu
22Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
NO WARRANTY
THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS" BASIS CARNEGIE MELLON UNIVERSITYMATERIAL IS FURNISHED ON AN AS IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holderrights of the trademark holder.
This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].
This work was created in the performance of Federal Government Contract Number FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. TheEngineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at 252.227-7013.
23Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University
Acronyms
CDD: capabilities developmentdocument
CDR iti l d i i
ICD: initial capability documentIOC: initial operational capability
CDR: critical design reviewCOCOMS: combatant commandersCONOPS: concept of operationsDAU D f A i iti U i it
IOT&E: operational test and evaluationIPT: integrated product teamIT: information technology
DAU: Defense Acquisition UniversityDSB: Defense Science BoardDoD: Department of DefenseDT d l t l t t
KPP: key performance parameterLRIP: low rate initial productionOT: operational test
DT: developmental testEVMS: earned value management
systemFOC: full operational capability
PDR: preliminary design reviewPMO: program management officeSME: subject matter expert
FOC: full operational capabilityFRP: full rate production
TRD: technical requirements document
24Finding Discipline in Agile AcquisitionOberndorf et al, 18 May 2011© 2011 Carnegie Mellon University