1© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management
Concept
Version 4.1 Executive Level
Mike Bloom and Joe Duquette
October 2001
Organization: ESC / EN (MITRE)
2© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management
““And through all this welter of change and And through all this welter of change and development, your mission remains fixed,development, your mission remains fixed,determined, inviolable -- it is to win our wars.”determined, inviolable -- it is to win our wars.”
General Douglas MacArthurGeneral Douglas MacArthur
3© Copyright 2001 The MITRE Corporation
“If you always do what you’ve always done, you’ll continue to get what you always got” and we can no longer afford it
• Only 16% of all Information Technology (computer and software) projects complete on time and on budget• 31% are cancelled before completion• The remaining 53% are late and over budget, with the typical cost growth exceeding the original budget by more
the 89%– Average overrun of project budgets was 189% – The average schedule overrun for projects that were in difficulty was 222%
• Of the IT projects that are completed, the final product contains only 61% of the originally specified features• If no formal systems engineering effort is included, projects run the risk of 50% to 100% development cost
overruns• DSB recommendation: Employ developers who demonstrate CMM Level 3 or equivalent. Certification must be
less that 2 years old. Eliminate the “escape clause”
“Charting the Seas of Technology: The CHAOS Study”
The Standish Group, January 1995Report of the Defense Science Board Task Force on Defense Software,
Nov 2000 INCOSE Systems Engineering Handbook
CMMI Based Process Improvement Risk Management Reasons
4© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management Process Improvement Objectives
Establish a Standard Risk Management Process Establish a Standard Risk Management Process That Will Assist in Achieving Overall Center ObjectivesThat Will Assist in Achieving Overall Center Objectives
ESC ObjectivesESC Objectives
Address Changes in Major Stakeholders
Address Acquisition and Operational Risks
Consistency with Current AF and DOD Policy
Tool Independent
Program Office Perspective
Value-Add Integral to Everyday Program
Management
Accessible by Sponsor via Web Technology
Address Risk to the Enterprise
Standard Risk ManagementStandard Risk ManagementProcess ObjectivesProcess Objectives
Shorter Time to Market
Integrated Command and Control and Combat Support
Harmonize Capabilities, Interoperability, User Needs, Budget, and Technology
Dealing With Uncertainty
Life-Cycle Systems Engineering
Streamline Communications
5© Copyright 2001 The MITRE Corporation
Risk Management is a continuous, forward-looking process that is an important part of business and technical management processes. Risk management needs to address issues that could endanger critical objectives. A continuous risk management approach is applied to ensure effective anticipation and mitigation of risks with critical impact across the project life cycle. CMMI Continuous Representation, V1.0, August 2000
Risk Management is a continuous, forward-looking process that is an important part of business and technical management processes. Risk management needs to address issues that could endanger critical objectives. A continuous risk management approach is applied to ensure effective anticipation and mitigation of risks with critical impact across the project life cycle. CMMI Continuous Representation, V1.0, August 2000
CMMI Based Process Improvement CMMI Risk Management Goals and Practices
DetermineRisk
Sourcesand
Categories
DefineRisk
Parameters
Establisha Risk
ManagementStrategy
DevelopRisk
MitigationPlans
ImplementRisk
MitigationPlans
Plan and Prepare for Risk Management Identify andAnalyze Risks
Risk Responsibility
Mitigate RisksFrom Project Planningand Project Monitoring
and Control
DAR
IdentifyRisks
Evaluate,Classify, and
PrioritizeRisks
6© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management Defined
“Risk is a measure of the inability to achieve system life cycle objectives ...” † such as the following:
• Assurance of Program Viability
• Provision of Operational Capability
• Delivery Within Negotiated Baseline
• Assurance of Operational Asset Survival
• Assurance of Mission Success
• Assurance of Personnel Safety and Performance
• Assurance of Integration with Operational Environment
“Risk has two components:• The probability (or likelihood) of failing
to achieve particular system life cycle objectives• The consequences of failing
to achieve those objectives” †
† Adapted from AFMCP 63-101, 9 July 1997
Risk “Risk is a measure of the inability to achieve system life cycle objectives ..” † such as the following:
•Assurance of Program Viability•Provision of Operational Capability•Delivery Within Negotiated Baseline•Assurance of Operational Asset Survival•Assurance of Mission Success•Assurance of Personnel Safety and Performance•Assurance of Integration with Operational Environment
“Risk has two components:•The probability (or likelihood) of failing to achieve particular system life cycle objectives•The consequences of failing to achieve those objectives” †
7© Copyright 2001 The MITRE Corporation
Systems Acquisition(Engineering and Manufacturing
Development, Demonstration, LRIP and Production)
Concept &Technology
Development
A
System Development& Demonstration
B
Production & Deployment
OT&E
FRPDecisionReview
CIOC
Operations &Support
FOC
Pre - SystemsAcquisition
SustainmentS y s t e m s A c q u i s i t i o n
( E n g i n e e r i n g a n d M a n u f a c t u r i n gD e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &
P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Based Process Improvement Program Life Cycle and Risk Management
8© Copyright 2001 The MITRE Corporation
Assurance of Program ViabilityAssurance of Program Viability
Provision of Operational CapabilityProvision of Operational Capability
Delivery Within Negotiated BaselineDelivery Within Negotiated Baseline
Assurance of Operational Asset SurvivalAssurance of Operational Asset Survival
Assurance of Mission SuccessAssurance of Mission Success
Assurance of Personnel Safety and PerformanceAssurance of Personnel Safety and Performance
Assurance of Integration with Ops. EnvironmentAssurance of Integration with Ops. Environment
ConceptDef.
Acq.Strat.
PackageDevelop
MissionInteg. O&S
CMMI Based Process Improvement Objectives and the System Life Cycle
System Life CycleSystem Life CycleObjectivesObjectives
9© Copyright 2001 The MITRE Corporation
ConceptDefinition
AcquisitionStrategy
PackageDevelopment
MissionIntegration
Operations &Support
Certification Congress Contractor Customer Deployment DOD ESC Experimentation Industry Legal /Contracting
SAF & HQ USAF SafetyCommunity
Sustainment Test Community Training User
CMMI Based Process ImprovementProcess Life Cycle Context
Players in the Life of a SystemWhen Do They Play
Key: = Must be involved = May be involved
10
© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Program Life Cycle and Risk Management
PPBS
Program Planning IPT
Concept DefinitionDefinition Process Modeling
Vision & Goals
Experimentation
Spiral Feedback
Threat Change
Mission Shift
Mission AreaPlanning
CapstoneArchitecture
Shortfalls &Opportunities
Budgeting & Tradeoffs
Operational &Technical
Architecture(s)Existing Direction Requirement Feasibility
Funding & Direction
User RequirementsDefinition
Operations & SupportOperations & Support
Phase Out & DisposalPhase Out & DisposalPhase Out & Disposal
Form UserGroup
BeginOperations
TCTOs, Reprocurement, Modifications,SLEP Engineering, Mission Shift
IOCIOCIOC
BeginSustainment
Operateand
Maintain
Deficiency ReportsIncident ReportsMaint Data Analysis
O&M Planning
Mission IntegrationMission Integration
IOCIOCIOC
Form Test & Integration
Working Group
Certification andAccreditation
Establish TestEnvironments
ProductionDecision
Yes
Back toDevelopment
Finalize ProductionReadiness
Test AssetsTest AssetsTest Assets
Security Net WorthinessInteroperability
LabsSupport ElementsTest AssetsTraining
Conduct Training, Connectivity,Installation, Deployment
OT&ESystem Fixes
InfrastructureReadiness
No
OT&E
Systems Acquisition(Engineering and Manufacturing
Development, Demonstration, LRIP and Production)
Concept &
TechnologyDevelopment
A
System Development& Demonstration
B
Production &
Deployment
OT&EFRPDecisionReview
CIOC
Operations &Support
FOC
Pre - SystemsAcquisition
Sustainment
Syst
em L
ife
Cyc
le
ConceptDefinition
AcquisitionStrategy
PackageDevelopment
MissionIntegration
Operations &Support
Funding &
Direction
ContractAward
DT&EComplete
DepotStart
System
Disposal
Concept &
TechnologyDevelopment
A
System Development& Demonstration
B
Production &
Deployment
OT&EFRPDecisionReview
CIOC
Operations &Support
FOC
ORDFunding & Direction
Acquisition StrategyAcquisition Strategy
Form Program Office
ASP
Source Selection
ContractAward
ContractContractAwardAward
SRD/TRDDevelopment
RequirementsFeasibility
RFPPreparation
Acquisition StrategyDevelopment
Package DevelopmentPackage Development
DT&E
ContractAward
ContractContractAwardAward
Post AwardConference
Form Working Level
IPTProduct IntegrationProduct Design
& Development
Define Post Award Work Packages and IPTs Fielding
DecisionPartial
Full
Back toDevelopment
ProductionReadiness
Test AssetsTest AssetsTest Assets
RiskZone
RiskZone
RiskZone
RiskZone Risk
Zone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone
RiskZone Risk
Zone
RiskZone
RiskZone
RiskZone
RiskZone
Risk
ZonesS y s t e m s A c q u i s i t i o n
( E n g i n e e r i n g a n d M a n u f a c t u r i n gD e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &
P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
11
© Copyright 2001 The MITRE Corporation
Systems Acquisition(Engineering and Manufacturing
Development, Demonstration, LRIP &Production)
ConceptDefinition
AcquisitionStrategy
PackageDevelopment
MissionIntegration
Operations &Support
OT&E
FRPDecisionReviewConcept &
TechnologyDevelopment
System Development& Demonstration
Production & Deployment Operations &
Support
FOC
Pre - SystemsAcquisition
Sustainment
IOC
BLOCK 2
BLOCK 3
A B C
CMMI Based Process Improvement Risk Management Space
Operational Risk Management
Acquisition Risk Management
12
© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management Process
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
13
© Copyright 2001 The MITRE Corporation
Systems Acquisition(Engineering and Manufacturing
Development, Demonstration, LRIP &Production)
ConceptDefinition
AcquisitionStrategy
PackageDevelopment
MissionIntegration
Operations &Support
OT&E
FRPDecisionReviewConcept &
TechnologyDevelopment
System Development& Demonstration
Production & Deployment Operations &
Support
FOC
Pre - SystemsAcquisition
Sustainment
IOC
BLOCK 2
BLOCK 3
A B C
Operational Risk Management
Acquisition Risk Management
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
No (Continue Monitoring)
Isthe Risk Plan
Working?
Yes(Continue
Monitoring) KeyMilestone
Approaching?
“n” MoSince Last Assessment
?
Revise Risk Plan
Step 7Monitor
Handling Plans
Step 5Establish Handling
Plans
Step 6Implement RiskHandling Plans
Step 1Prepare
Step 2Identify
Risks & Hazards
Step 4Decide on Control
Options
Step 3Assess & Prioritize
Risks
New Phase or KeyStakeholder
?
Yes
No
No (Continue
Monitoring)
Yes
Yes
(Continue Monitoring)
No
CMMI Based Process Improvement System Life Cycle Application of Risk Process
Applied Continuously Through Life Cycle
14
© Copyright 2001 The MITRE Corporation
Risk Management Becomes a Management PriorityRisk Management Becomes a Management Priority• Manager Becomes Advocate of Risk ManagementManager Becomes Advocate of Risk Management• Manager Commits Energy and Resource to EffortManager Commits Energy and Resource to Effort
Risk Management Becomes a Program PriorityRisk Management Becomes a Program Priority• Stakeholders Become Co-Sponsors Stakeholders Become Co-Sponsors • Stakeholders Commit to Sufficient ResourceStakeholders Commit to Sufficient Resource
Risk Management Becomes a Mission PriorityRisk Management Becomes a Mission Priority• Process is Focused on Successful MissionProcess is Focused on Successful Mission• Stakeholders Become Familiar with Program and MissionStakeholders Become Familiar with Program and Mission
Stakeholders Identify Mission UncertaintiesStakeholders Identify Mission Uncertainties• Risk Manager Makes Various Risk Data and Information Available to All StakeholdersRisk Manager Makes Various Risk Data and Information Available to All Stakeholders• Each Stakeholder Formulates Individual Concerns/UncertaintiesEach Stakeholder Formulates Individual Concerns/Uncertainties
Action 1:Obtain Buy-In From
Program Manger on RiskAssessment and Management
Action 3:Identify and Distribute Key
Program/MissionObjectives & Requirements
Action 2:Identify & Notify Key Program/
Mission Stakeholders
Action 4:Identify, Review, and Distribute
Applicable Risk /Hazard Taxonomies
CMMI Based Process Improvement Risk Management Process
Step 1 - PrepareStep 1 - Prepare
CommitCommit Form the TeamForm the Team Know the MissionKnow the Mission Think RisksThink Risks
PM
Buy
-In
PM
Buy
-In
Stakeholder Buy-IN
Stakeholder Buy-IN
Mission Priority
Mission Priority
ID RisksID Risks
15
© Copyright 2001 The MITRE Corporation
aConduct Risk Management Meetings Conduct Risk Management Meetings • Initial Meeting Sets Tone and Opens Channels of CommunicationInitial Meeting Sets Tone and Opens Channels of Communication• Subsequent Meetings Need to be Held as Program ProgressesSubsequent Meetings Need to be Held as Program Progresses
Understand Mission, How Risk Will be Managed, and Tools AvailableUnderstand Mission, How Risk Will be Managed, and Tools Available• Compare Current Mission with Past Missions (Taxonomies)Compare Current Mission with Past Missions (Taxonomies)• Make Sure Everyone Understands How Risk Management Will be DoneMake Sure Everyone Understands How Risk Management Will be Done
Identify Program RisksIdentify Program Risks• The Inability to Achieve Program ObjectivesThe Inability to Achieve Program Objectives• When will it HappenWhen will it Happen
Classify the RisksClassify the Risks• Can Use a Predefined Structure (Taxonomy, WBS, CPT etc.)Can Use a Predefined Structure (Taxonomy, WBS, CPT etc.)• Can Use a Self-Organized StructureCan Use a Self-Organized Structure
Consolidate Like Risk and Write Risk StatementsConsolidate Like Risk and Write Risk Statements• Capture Concise Description to be Acted UponCapture Concise Description to be Acted Upon• Risk Statement = Condition + ConsequenceRisk Statement = Condition + Consequence
Establish TeamEstablish Team Develop UnderstandingDevelop Understanding IdentifyIdentify WriteWrite
Action 1:Assemble
Stakeholders for Risk Assessment
Action 2:Review Program/Mission
Objectives, Taxonomies andRisk Assessment Process
Action 3:Conduct
Risk Identification
Action 4:Group
Related Risks
Action 5:Consolidate RelatedRisks & Write “If -
Then” Risk Statements
CMMI Based Process Improvement Risk Management Process
Step 2 - Identify the Risks and Hazards
ClassifyClassify
16
© Copyright 2001 The MITRE Corporation
A• Identify Consequences or Level of Impact to the Program If the Risk OccursIdentify Consequences or Level of Impact to the Program If the Risk Occurs
• Establish or use Predetermined Impact Categories Establish or use Predetermined Impact Categories (e.g. Critical, Serious, Moderate, Minor, Negligible)(e.g. Critical, Serious, Moderate, Minor, Negligible) • Determine the Probability of OccurrenceDetermine the Probability of Occurrence
• Establish or Use Predetermined Probability Bands Establish or Use Predetermined Probability Bands ( e.g. Very Unlikely, Unlikely, Probably, Likely, Very Likely)( e.g. Very Unlikely, Unlikely, Probably, Likely, Very Likely)
• For Each Risk Identify the Time Period When the Risk Is Likely to OccurFor Each Risk Identify the Time Period When the Risk Is Likely to Occur• Establish or Use Predetermined Time Periods ( e.g. Near, Midterm, Far)Establish or Use Predetermined Time Periods ( e.g. Near, Midterm, Far)
• Incorporate Existing Identified Risks With Newly Identified RisksIncorporate Existing Identified Risks With Newly Identified Risks• Reassess Existing Risks Following Actions 1,2, and 3.Reassess Existing Risks Following Actions 1,2, and 3.• Fold Existing Risks and Newly Identified Risk TogetherFold Existing Risks and Newly Identified Risk Together
• Prioritize RisksPrioritize Risks• Involves Grouping Risks Using Impact, Probability and TimingInvolves Grouping Risks Using Impact, Probability and Timing• Objective Is to Identify Most Serious Program RisksObjective Is to Identify Most Serious Program Risks
• Identify Risk Handling BandsIdentify Risk Handling Bands• Place Risks in to Appropriate Handling BandPlace Risks in to Appropriate Handling Band• Objective Is to Establish Preliminary Resource ConstraintsObjective Is to Establish Preliminary Resource Constraints
Impact ?Impact ? Probability ?Probability ? Old Risks ?Old Risks ? Coarse SortCoarse Sort
Action 1:Identify & Get
Consensus on Impact / Severity for Each Risk
Action 2:Identify & Get Consensus
on Probability ofOccurrence for Each Risk
Action 3:Identify Time Window when
Risk Could Occur
Action 5:Prioritize Risks
by Impact,Probability & Time
Action 6:Identify Handling
Bands
Action 4:Reassess AnyExisting Risks
in Database
CMMI Based Process Improvement Risk Management Process
Step 3 - Assess and Prioritize Risks
When ?When ? PrioritizePrioritize
17
© Copyright 2001 The MITRE Corporation
a
Choose Risk Handling Options Choose Risk Handling Options
Decide Which Risk Will:Decide Which Risk Will:• Be AssumedBe Assumed• Be Watched (set “Triggers” or “Cues”)Be Watched (set “Triggers” or “Cues”)• AvoidedAvoided• TransferredTransferred• MitigatedMitigated
Assign Responsibility for Risk PlanningAssign Responsibility for Risk Planning• Avoid Risk - Research, Design, Fund etc.Avoid Risk - Research, Design, Fund etc.• Transfer Risk - To Whom, AcceptanceTransfer Risk - To Whom, Acceptance• Mitigate Risk - Strategy, Resources etc.Mitigate Risk - Strategy, Resources etc.
Establish and Update a Risk DatabaseEstablish and Update a Risk Database
OptionsOptions Easy RisksEasy Risks ResponsibilityResponsibility CaptureCapture
Action 1:Identify Handling Options
Within Each Risk Band
Action 2:Identify Which
Risks will be Assumed or Watched
Action 3:Identify Which
Risks will be Avoided,Transferred or Mitigated
Action 4:Assign Plan OPRs
for Avoided, Transferred, or Mitigated Risks (Active)
Action 5:Establish orUpdate Risk
Database
CMMI Based Process Improvement Risk Management Process
Step 4 Decide on Handling Options
High Medium Low
Ranked Risks
Assume
AvoidTransferMitigateMonitor
AvoidTransferMitigateMonitor
“Cues”ActionPlans
“Triggers”ContPlans
Hard RisksHard Risks
18
© Copyright 2001 The MITRE Corporation
ADraft the Handling PlansDraft the Handling Plans• Avoided, Transferred and Mitigated RisksAvoided, Transferred and Mitigated Risks• Contingency and Risk Status Change PlansContingency and Risk Status Change Plans• 1-3 Pages, Standard Format, Matches Database1-3 Pages, Standard Format, Matches Database
Program Manager Review and ApprovalProgram Manager Review and Approval• Program Manager Buy-in of the Handling PlanProgram Manager Buy-in of the Handling Plan• Formal Process to Insure That Resources Required Are AllocatedFormal Process to Insure That Resources Required Are Allocated• Opportunity to Improve the Handling Plan and Provide Team PerspectiveOpportunity to Improve the Handling Plan and Provide Team Perspective• Process Is Iterative and May Require a Number of Changes to Proposed PlansProcess Is Iterative and May Require a Number of Changes to Proposed Plans• Can Provide an Opportunity to Expose and Adjudicate Different Points of ViewCan Provide an Opportunity to Expose and Adjudicate Different Points of View
Funded, Directed and Integrated with Program ManagementFunded, Directed and Integrated with Program Management• Usually Requires Expenditure of Resources (E.G. Cost Estimates And/or Budget Actions)Usually Requires Expenditure of Resources (E.G. Cost Estimates And/or Budget Actions)• For a Handling Plan to Have Impact It Must Be EnforceableFor a Handling Plan to Have Impact It Must Be Enforceable• Appropriate Changes to Program Directives and Execution Documents and MonitoredAppropriate Changes to Program Directives and Execution Documents and Monitored
Develop Plans & EstimatesDevelop Plans & Estimates Review and ApproveReview and Approve Fund, Direct, IntegrateFund, Direct, Integrate
Action 1:Develop Draft Handling
Plans and Associated Resource Requirements
Action 2:Program Manager Reviewand Approval of Handling
Plans
Action 3:Handling Plan are Funded,
Directed, and Integratedwith Program Management
CMMI Based Process Improvement Risk Management Process
Step 5 - Establish Handling Plans
DRAFT
Approved
Integrated
19
© Copyright 2001 The MITRE Corporation
aComplete Risk Management Plan (RMP) Complete Risk Management Plan (RMP) • RMP Can Be Completed - the Program Now Has a Good Understanding of Program Risk RMP Can Be Completed - the Program Now Has a Good Understanding of Program Risk • Risk Management Risk Management Program Management Program Management
Provide for a Mechanism to MonitorProvide for a Mechanism to Monitor• Handling PlansHandling Plans• Triggers and CuesTriggers and Cues
Implement the Handling PlansImplement the Handling Plans• Implement = Knowledge + Resources + Authority to ActImplement = Knowledge + Resources + Authority to Act• Communicate, Communicate, CommunicateCommunicate, Communicate, Communicate
Finalize RMPFinalize RMP Monitoring ApproachMonitoring Approach ImplementImplement Monitor ProgressMonitor Progress
Action 3:Implement Handling Plans as
Authorized, Funded, & Scheduled Work with Exit Criteria
Action 4:Provide Reporting on Handling
Plan Results & Progress in Meeting Exit Criteria
CMMI Based Process Improvement Risk Management Process
Step 6 - Implement Risk Handling
Action 2:Provide Mechanism to
Monitor Triggers, Cues and Handling Plans
Action 1:Finalize Risk Management Plan
& Management Infrastructure
RiskHandling
Handling Status Review …. Handling Status Review …. TAKE TAKE ACTION!ACTION!
Risk Management Database …. Risk Management Database …. UPDATE!UPDATE!
20
© Copyright 2001 The MITRE Corporation
a
YesYes
No “Accept”Risk
?
Prepare, Approveand ImplementHandling Plan
No
MonitorHandling
Plans
Time toReassess
?
No
No
Yes
Yes
No
Yes
“Accept”Risk
?
Est. Trigger &Contingency
Plan
Yes
Yes
Time toReassess
?
No
No Yes
Doneor
OBE?
Yes
Yes
Work The“Issue”
Yes
Yes
No
Yes
No
No NoLowLow
Handling BandAccept Do Nothing Until Reassessment
HighHighHandling
Band?
PeriodicReassessment
EstablishHandling
Bands
MonitorTrigger
TriggerOccurs
?
ImplementContingency
Plan
Continue Implementation of Contingency Plan
STOP
Is ItWorking
?
MediumMediumHandling
Band?
Is ItWorking
?
Doneor
OBE?
STOP CueOccurs
?
MonitorCues
ReassessIndividual Risk
Immediately
Establish Cuesfor
Increasing Risk
Time toReassess
?
TriggerMonitoring
Handling PlanMonitoring
CueMonitoring
Yes
TriggerTriggerMonitoringMonitoring
Handling Handling PlanPlan
MonitoringMonitoring
CueCueMonitoringMonitoring
CMMI Based Process Improvement Risk Management Process
Step 7 - Monitor Handling Plans
Action 1:Periodically Review
Handling Plan Results
Action 2:Stop or Modify Handling Plans
and Resources, if required
Action 3:Retire Risks WhenHandling Plans are
Successfully Completed
Action 4:Update Risk Database for Handling Plan Progress
& Risk Retirement
ReviewReview Modify or StopModify or Stop RetireRetire Update DatabaseUpdate Database
21
© Copyright 2001 The MITRE Corporation
CMMI Based Process Improvement Risk Management Process
Step 7 - Monitor Handling Plans
There may be other reasons or events that will require a reworking of the process …….. it is imperative that the program team be open to this possibility and be prepared to fix the problem because …… Risk Management Risk Management Program Management Program Management
KeyMilestone
Approaching?
Return toStep 2
Yes
NewPhase or KeyStakeholder
?
Return toStep 1
Yes
Is This Not Working
?
Return to Step 1Revise Risk Plan
Yes
STEP 7 - MONITOR HANDLING PLANS
Action 1:Periodically Review Action Plan Results
Action 2:Stop or Modify Action Plansand Resources, if required
Action 3:Retire Risks WhenAction Plans are
Successfully Completed
Action 4:Update Risk Database for
Action Plan Progress & Risk Retirement
“n” MoSince Last Assessment
?
Return toStep 2
Yes
“n” MoSince Last Assessment
?
Return toStep 2
Yes
Step 7 DecisionsStep 7 Decisions
JAN
FEB
MAR
APR
MAYJUN
JUL
AUG
SEP
OCT
NOVDEC
Establish A Regular Review Cycle & Go Back Through the
Process Starting With Step 2
NewPhase or KeyStakeholder
?
Return toStep 1
Yes
Step 7 DecisionsStep 7 Decisions
Pre ORD
Post ORD
Dev. Period
Ops. Env.Integ.
O&M
New Stakeholders Are Added or Changed Throughout the Program Life Cycle
KeyMilestone
Approaching?
Return toStep 2
Yes
Step 7 DecisionsStep 7 Decisions
PROACTIVE LOOK AHEAD•To Review & Identify Risks•To Create New Handling Plans•Return to Step 2 to Insure all Risk and Hazards Have Been Identified
CSystem
IntegrationSystemDemo
System Dev & Demonstration
IOCA B BA
Review Review
LRIP
Support
Continuous communication with usersEarly & continuous testing
BLOCK II
BLOCK III
Single Step orEvolution to Full
Capability
ConceptExploration
ComponentAdvanced
Development
Concept & Tech Development
Review
Rate Production &Deployment
Production & Deployment
CommunicateCommunicateMajor ProgramMajor ProgramRisk to SeniorRisk to Senior
Decision MakerDecision Maker
Rebuild RMP &Rebuild RMP &Re-Establish Buy-InRe-Establish Buy-In
of Present Set ofof Present Set ofStakeholdersStakeholders
Return to Step 1 and Revise theRisk Management Plan
Is This Not Working
?
Return to Step 1Revise RMP
Yes
Step 7 DecisionsStep 7 Decisions
22
© Copyright 2001 The MITRE Corporation
“Life is tough, but it’s tougher if you’re stupid”
John WayneJohn Wayne
asas
Sergeant John M. Stryker, USMC, Sergeant John M. Stryker, USMC,
inin““The Sands of The Sands of
Iwo Jima”Iwo Jima”
CMMI Based Process Improvement Risk Management Process
23
© Copyright 2001 The MITRE Corporation
S y s t e m s A c q u i s i t i o n( E n g i n e e r i n g a n d M a n u f a c t u r i n g
D e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Process Life Cycle ContextProgram Life Cycle
Concept DefinitionConcept Definition
Vision & GoalsJV 2020JV 2020
ExperimentationEFX, ATD, ACTD
Spiral Feedback
Threat ChangeSTARSTAR
Mission ShiftEAFEAF
Existing Direction
ORD
Process Modeling
CapstoneArchitecture
Shortfalls &Opportunities
PPBS
Budgeting & Tradeoffs
Requirement Feasibility
User RequirementsDefinition
Program DirectionProgram DirectionProgram DirectionProgram Direction
Mission AreaPlanning
Operational &Technical
Architecture(s)
Program Planning IPT
24
© Copyright 2001 The MITRE Corporation
S y s t e m s A c q u i s i t i o n( E n g i n e e r i n g a n d M a n u f a c t u r i n g
D e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Process Life Cycle ContextProgram Life Cycle
Form Program Office
SRD/TRDDevelopment (1)
RequirementsFeasibility
PMDAPB
M edica l
B usiness A pp lica tions
G loba l C om batS upport S ystem (G C S S )
S oftw are D istribu tionfrom C entra l F iles
W eb S erv ices
M egacenter S erv ices
E lectron ic M a il D e live ry
S A TC O M
C om m erc ia l F iber
M S S
R F N ets
Teleports
W ire lessC om m
D octrineP olicy
E ngineering
StandardsA rch itectures
Spectrum
G overnance
D IS N
M edica l
B usiness A pp lica tions
G loba l C om batS upport S ystem (G C S S )
S oftw are D istribu tionfrom C entra l F iles
W eb S erv ices
M egacenter S erv ices
E lectron ic M a il D e live ry
S A TC O M
C om m erc ia l F iber
M S S
R F N ets
Teleports
W ire lessC om m
D octrineP olicy
E ngineering
StandardsA rch itectures
Spectrum
G overnance
D IS N
Ops & TechArchitectures
UserRequirements
Funding
Acquisition StrategyAcquisition StrategyM e d ica l
B u s in e ss A pp lica tio n s
G lo b a l C o m b a t
S u p p ort S ys tem (G C S S )
S o ftw a re D is trib u tio nfro m C e n tra l F ile s
W e b S e rv ices
M e g a cen te r S e rv ice s
E le c tro n ic M a il D e live ry
S A T C O M
C o m m erc ia l F ib e r
M S S
R F N e ts
T e le p o rts
W ire le ssC o m m
D octrin eP o licy
E n gin eerin g
S tan d ard sA rch itectu res
S p ectru m
G overn an ce
D IS N
ArchitectureCapstone
RFPPreparation (2)
Acquisition StrategyDevelopment
ASP
Source Selection
ContractContractAwardAward
ContractContractAwardAward
25
© Copyright 2001 The MITRE Corporation
S y s t e m s A c q u i s i t i o n( E n g i n e e r i n g a n d M a n u f a c t u r i n g
D e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Process Life Cycle ContextProgram Life Cycle
ContractContractAwardAward
ContractContractAwardAward
Post AwardConference
SystemArchitectureDevelopment
Define Post Award Work Packages and IPTs
Form Working Level
IPT
Package DevelopmentPackage Development
Product Integration
DT&E
Partial
Full
Back toDevelopment Test
Ready?
TestTestArticlesArticlesTestTest
ArticlesArticles
ProductionReadiness
Package Design & Development
Package Design & DevelopmentPackage Design & Development ProductIntegration
RequirementsRefinement
Package Consensusand Contract Action
Spiral “n”Design
Package Tradeoffand Decision
Spiral “n”Development
26
© Copyright 2001 The MITRE Corporation
S y s t e m s A c q u i s i t i o n( E n g i n e e r i n g a n d M a n u f a c t u r i n g
D e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Process Life Cycle ContextProgram Life Cycle
Mission IntegrationMission Integration
Form Test & Integration
Working Group
TestTestArticlesArticlesTestTest
ArticlesArticles
Establish TestEnvironments
Certification andAccreditation
OT&E
ProductionDecision
Back toDevelopment
No
OT&ESystem Fixes
Yes
Finalize ProductionReadiness
Conduct Training, Connectivity,Installation, Deployment
InfrastructureReadiness
DepotDepotStartStart
DepotDepotStartStart
27
© Copyright 2001 The MITRE Corporation
S y s t e m s A c q u i s i t i o n( E n g i n e e r i n g a n d M a n u f a c t u r i n g
D e v e l o p m e n t , D e m o n s t r a t i o n , L R I P &P r o d u c t i o n )
Sys
tem
Lif
e C
ycle
A d v o c a c y& P l a n n i n g
A c q u i s i t i o nS t r a t e g y
P a c k a g eD e v e l o p m e n t
M i s s i o nI n t e g r a t i o n
O p e r a t i o n s &S u p p o r t
F u n d i n g&
D i r e c t i o n
C o n t r a c tA w a r d
D T & EC o m p l e t e
D e p o tS t a r t
C o n c e p t &T e c h n o l o g y
D e v e l o p m e n t
A
S y s t e m D e v e l o p m e n t& D e m o n s t r a t i o n
B
P r o d u c t i o n & D e p l o y m e n t
O T & E
F R PD e c i s i o nR e v i e w
C
O p e r a t i o n s &S u p p o r t
F O C
P r e - S y s t e m sA c q u i s i t i o n
S u s t a i n m e n t
S y s t e mD i s p o s a l
I O C
B L O C K 2
B L O C K 3
CMMI Process Life Cycle ContextProgram Life Cycle
Form UserGroup
Operations & MaintenanceOperations & MaintenanceIOCIOCIOCIOC
Phase Out & DisposalPhase Out & DisposalPhase Out & DisposalPhase Out & Disposal
BeginOperations
BeginSustainment
Operateand
MaintainO&M Planning
Spiral N+1 (e.g. TCTOs, Reprocurement, Modifications, SLEP Engineering, Mission Shift)