29
1 DA HIEF YSTEMS NGINEER ASN (RDA) Chief Systems Engineer DoN PoPS to ICPM Core Common Metrics and OSD DAPS NDIA ICPM 16 November 2009
| Date post: | 13-Jan-2016 |
| Category: |
Documents |
| Upload: | martha-blake |
| View: | 214 times |
| Download: | 0 times |
1
DA HIEF
YSTEMS NGINEER
ASN (RDA)Chief Systems Engineer
ASN (RDA)Chief Systems Engineer
DoN PoPS to
ICPM Core Common Metrics and OSD DAPS
NDIA ICPM
DoN PoPS to
ICPM Core Common Metrics and OSD DAPS
NDIA ICPM
16 November 2009
2
DA HIEF
YSTEMS NGINEER
DoN Probability of Program Success (PoPS)to ICPM Core Common Metrics and
Defense Acquisition Program Support (DAPS)
Requested to compare/contrast ICPM and PoPS
Also looked at DAPS to PoPS
3
DA HIEF
YSTEMS NGINEER
3
Program Health
Factors
Metrics
Criteria
Program Requirements
Program Resources
Program Planning/Execution
External Influencers
Parameter Status
Scope Evolution
CONOPS
Budget and Plan
Manning
Acquisition Management
Sustainment
Industry/Company Assessment
Software
Total OwnershipCost Estimating
Government Program Office Performance
Test and Evaluation
Contract Planning and Execution
Technical Maturity
Fit in Vision
Program Advocacy
Interdependencies
Naval PoPS 2.0
Technology Protection
Naval Probability of Program SuccessNaval Probability of Program Success (PoPS) v2.0 (PoPS) v2.0
6 October 2008 PDASN Memo Naval PoPS Guidance:
o Required for all DoN Gate Reviews and any time program health is discussed
o Standard DoN method of representing the health of all ACAT Programs and any other programs subject to the DoN Acquisition process
o Use the methods and tools described in the Naval PoPS Guidebook and supplemental Handbooks
Consistent means to assess program health, criteria adjusts as Program matures
Used for Portfolio Assessments and identification of Department wide systemic issues
PoPS Criteria Scoring (Example)
4
DA HIEF
YSTEMS NGINEER
4
ICPM Core Common Metrics
Under Development by NDIA ICPM
Shared view of program performance by customers and industry
Establishes critical behaviors / techniques for program success
Portfolio management
Metrics
Program Health
FactorsRequirements
Stability &
Completeness
Predictive Measures of
Program Performance
Critical Skills
Requirements Volatility
TBD/TBR Burn down
Risk / Opportunity Summary
Funding Stability &
Contract Health
Technical Performance &
Productivity Variance
Risk & Opportunity
Management
Program Cost &
Schedule
Staffing & Critical
Skills
Program Funding Plan
Program Funding Status
Contract Change Volume
TPM (Summary quick look)
TPM Progress/Regress Burn Down Chart
Software Productivity
Cost Performance Index
Schedule Performance Index
Cum CPI vs. TCPIEAC
Baseline Execution Index
Requirements Traceability
Risk / Opportunity relative to MR
Risk Mitigation Plan
Late/On-time Starts
Critical Path
Staffing Profile
Supply Chain
Performance
Parts Demand Fulfillment
Supplier Acceptance Rate
Supplier Late Starts
Top 8 IT Readiness Indicators
Resources
Physical Inventory, Maintenance, Records, Relief of Stewardship
Action Tracking / Burndown
Issues/Help
Needed
Customer
Satisfaction
CPAR
Award Fee
Product Performance
Transparency of data to avoid surpriseso Clear Understanding of Requirementso Transparency of Technical Performanceo Cost / Schedule Performance, Risk Management,
Resources, and Supplier Performanceo Help Needed Culture – Across All Program Lines and
Open Communication Relative to Issueso Clear, Continuous Feedback from the Customer
Categories
5
DA HIEF
YSTEMS NGINEER
Defense Acquisition Program Support (DAPS)
Defines the PSR approach to assess defense system development programs by the OSD Program Support Team.o not a checklist to be followed rigidly, nor must each assessment address all
assessment areas and sub-areas listed. o Tailorable to ensure important areas are not overlooked and comprehensively
examinedo Includes an extensive list of programmatic and technical factors that are used to
build the review plan.
Mission
Capabilities /
Req’ts Generation
DoD DAPS
Readiness LevelsCONOPs
Analysis of Alternatives
Design Considerations
Resources ManagementTechnical
Process
Program
Performance
Special
Interest
Areas
Budget Sufficiency & Phasing
Staffing Level
Program Schedule Overview
Acquisition Strategy
Knowledge-based Decisions and
Milestones
Program & Project Management
Contracting
Effectiveness
Suitability
Survivability
Production
Capabilities
Requirements Development
Technical Baseline
Engineering Tools
Software
Design Verification
Supportability Planning
Program Assessment
Areas
Criteria
Sub-Areas
Pre-Milestone A
3.0 Management
3.1 Acquisition Strategy3.1.1 Credibility - The program manager (PM) is developing a credible Acquisition Strategy that will provide the basis for meeting program objectives and therefore will be an aid in gaining program acceptance and support.
3.1.1.Q1: How is the Acquisition Strategy realistic?
3.1.1.Q2: Does the program face any of the following pressures, which work against the realism of the Acquisition Strategy?
3.1.1.Q3: How is the Acquisition Strategy stable?
3.1.1.Q4: Are any of the potential causes of instability to the Acquisition Strategy present?
Etc.
6
DA HIEF
YSTEMS NGINEER
Key Assessment Areas / FactorsICPM versus PoPS version 2.0
ICPM
Categories
DoN PoPS Factors
Requirements Stability & Completeness
Program Requirements
Customer Satisfaction Program Planning & ExecutionTechnical Performance &
Productivity Variance
Risk & Opportunity Management
Program Cost & Schedule
Supply Chain Performance
Resources
Funding Stability & Contract Health
Program Resources
Staffing & Critical Skills
Issues/Help Needed
External Influencers
ICPM DoN PoPS10 Categories 4 Factors30 Metrics 18 Metrics
Criteria* Criteria*
*Particular Criteria and number is dependent on
Life Cycle Phase
All ICPM Categories and Metrics are included in PoPS Factors and Metrics
7
DA HIEF
YSTEMS NGINEER
ICPM DoN POPS
Requirements Stability and
Completeness
CONOPs Program RequirementsRequirements Volatility Scope Evolution
TBD/TBR Burn down
Requirements Traceability Parameter Status
Funding Stability & Contract Health
Program Funding Plan Total Ownership Cost Program Planning & ExecutionContract Change Volume Contract Execution
Program Funding Status Program Resources Factor- Budget
Technical Performance & Productivity Variance
TPM (Summary quick look) Technical Maturity Program Planning & ExecutionTPM Progress/Regress Burn Down Chart
Software Productivity Software
Risk and Opportunity Management
Risk/opportunity Summary Government Program Office Performance Program Planning & ExecutionRisk/Opportunity relative to MR
Risk Mitigation Plan
Program Cost and Schedule
Cost Performance Index Contract Planning and Execution Program Planning & ExecutionSchedule Performance Index
Statistical EACs
Cum CPI vs TCPIEAC
Baseline Execution Index
Late/On-Time Starts
Critical Path Acquisition Management
Summary ICPM versus PoPS
8
DA HIEF
YSTEMS NGINEER
ICPM DoN POPS
Staffing & Critical SkillsCritical Skills Manning Program ResourcesStaffing Profile
Supply Chain Performance
Parts Demand Fulfillment Contract Planning & Execution Program Planning & ExecutionSupplier Acceptance Rate
Supplier Late Starts
ResourcesTop 8 IT Readiness Indicators Software Program Planning & ExecutionPhysical Inventory, Maintenance, Records, Relief of Stewardship
Sustainment
Government Program Office Performance
Customer Satisfaction
CPAR Industry and Company Assessment Program Planning & ExecutionAward Fee Contract Planning and Execution
Product Performance Technical Maturity
Test and Evaluation
Program Requirements Factor – Parameter Status
Issues/Help NeededAction Tracking /Burndown Assessed at DoN SETRs and
Gate Reviews
Summary (cont.) ICPM versus PoPS
9
DA HIEF
YSTEMS NGINEER
Key Assessment Areas / FactorsDAPS versus PoPS version 2.0
OSD DAPS Areas
DoN PoPS Factors
Mission Capabilities / Requirements Generation
Program Requirements
Resources Program Resources
Management External Influences
Program Planning & ExecutionTechnical Process
Program Performance
Special Interest Areas – Technical Readiness
OSD DAPS DoN PoPS6 Areas 4 Factors
22 Sub-Areas 18 Metrics
59 Factors
-Criteria*•Questions*
-Criteria*
*Particular Criteria and number is dependent on
Life Cycle Phase
10
DA HIEF
YSTEMS NGINEER
OSD DAPS DoN POPSMission Capabilities / Requirements Generation
CONOPs CONOPs Program RequirementsAnalysis of Alternatives Scope Evolution
Capabilities Parameter Status
Resources Program Schedule Overview Acquisition Management Program Planning & ExecutionBudget Sufficiency & Phasing Total Ownership Cost Estimating
Budget Program ResourcesStaffing Level Manning
Management Acquisition Strategy Fit in Vision External InfluencesProgram Advocacy
Acquisition Management Program Planning & ExecutionKnowledge-Based Decisions & Milestones
Contracting Contract Planning & Execution
Industry / Company Assessment
Program & Project Management Government Program Office Performance
Interdependencies External Influences
Technical Process Design Considerations Program Planning & ExecutionRequirements Development Technical Maturity
Technical Baseline
Engineering Tools
Software Software
Design Verification Test and Evaluation
Supportability Planning Sustainment
Special Interest Areas Readiness Level Technical Maturity
Program Performance Effectiveness
Test and Evaluation
Suitability Sustainment
Survivability Test and Evaluation
Production Industry / Company Assessment
SummaryDAPS versus PoPS
DoN Gate Review
and DoN SETR
Process
11
DA HIEF
YSTEMS NGINEER
Summary ICPM and DoN PoPS
o ICPM appears to serve Executive Level discussions
o ICPM assessment factors similar to those in PoPS
o However differences in Criteria may be warranted ICPM – Health of Program from Contractor view PoPS – Health of Program from Government view
DAPS and DoN PoPSo DAPS is a mix of Executive and Management Level informationo DAPS factors covered by
DoN Gate Reviews Core Criteria and PoPS (Exec / Management Level) Systems Engineering Technical Reviews (SETRs) (Management / Detail Level)
DoN PoPSo Executive Level with input from Management / Detail Reviews (SETR)
o Used continuously for life of Program
o As input for Major Decision Reviews (Gate and Milestone)
o Applicable to All ACAT Programs
12
DA HIEF
YSTEMS NGINEER
Back-upBack-up
13
DA HIEF
YSTEMS NGINEER
Details of Core Criteria and Templates by Gate - URL: https://nserc.navy.mil/asnrda/asn_rda_pops/pops20dev/Step%202%20Draft%20Documents/Forms/AllItems.aspx?RootFolder=%2fasnrda%2fasn%5frda%5fpops%2fpops20dev%2fStep%202%20Draft%20Documents%2f0%20Draft%20Core%20and%20Briefing%20Templates%20in%20Exec%20Staffing&FolderCTID=&View=%7b7AEE306D%2d7B71%2d4326%2d999E%2dDEEC9A1DF241%7d
Details of PoPS Criteria and Templates by Gate - URL: https://nserc.navy.mil/asnrda/asn_rda_pops/pops20dev/Step%202%20Draft%20Documents/Forms/AllItems.aspx?RootFolder=%2fasnrda%2fasn%5frda%5fpops%2fpops20dev%2fStep%202%20Draft%20Documents%2fProposed%20PoPS%20v2%20Criteria%20and%20Templates%20in%20Exec%20Staffing&FolderCTID=&View=%7b7AEE306D%2d7B71%2d4326%2d999E%2dDEEC9A1DF241%7d
14
DA HIEF
YSTEMS NGINEER
ICPMRequirements Stability and Completeness Category
DoN POPSProgram Requirements Factor
Metric Metric
CONOPs
Requirements VolatilityScope Evolution
TBD/TBR Burn down
Requirements Traceability Parameter Status
Requirements Assessment ICPM versus PoPS
15
DA HIEF
YSTEMS NGINEER
ICPMFunding Stability & Contract Health Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
Program Funding Plan Total Ownership Cost Estimating
Contract Change Volume Contract Execution
Program Funding Status Program Resources Factor -
Budget
Resources Assessment ICPM versus PoPS
16
DA HIEF
YSTEMS NGINEER
ICPMTechnical Performance & Productivity Variance Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
TPM (Summary quick look)
Technical MaturityTPM Progress/Regress Burn Down Chart
Software Productivity
Software
Resources Assessment ICPM versus PoPS
17
DA HIEF
YSTEMS NGINEER
ICPMRisk & Opportunity Management Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
Risk / Opportunity Summary
Government Program Office PerformanceRisk / Opportunity relative to MR
Risk Mitigation Plan
Resources Assessment ICPM versus PoPS
18
DA HIEF
YSTEMS NGINEER
ICPMProgram Cost & Schedule Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
Cost Performance Index
Contract Planning and Execution
Schedule Performance Index
Statistical EACs
Cum CPI vs. TCPIEAC
Baseline Execution Index
Late/On-time Starts
Critical PathAcquisition Management
Resources Assessment ICPM versus PoPS
19
DA HIEF
YSTEMS NGINEER
ICPMStaffing & Critical Skills Category
DoN POPSProgram Resources Factor
Metric Metric
Critical Skills Manning
Staffing Profile
Resources Assessment ICPM versus PoPS
20
DA HIEF
YSTEMS NGINEER
ICPMSupply Chain Performance Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
Parts Demand Fulfillment
Contract Planning & ExecutionSupplier Acceptance Rate
Supplier Late Starts
Resources Assessment ICPM versus PoPS
21
DA HIEF
YSTEMS NGINEER
ICPMResources Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
Top 8 IT Readiness Indicators Software
Physical Inventory, Maintenance, Records, Relief of Stewardship
Sustainment
Government Program Office Performance
Resources Assessment ICPM versus PoPS
22
DA HIEF
YSTEMS NGINEER
ICPMIssues/Help Needed Category
DoN POPS
Metric Metric
Action Tracking / Burndown
Resources Assessment ICPM versus PoPS
Assessed at DoN Gate
Reviews and SETRs
23
DA HIEF
YSTEMS NGINEER
ICPMCustomer Satisfaction Category
DoN POPSProgram Planning & Execution Factor
Metric Metric
CPAR Industry and Company Assessment
Award Fee Contract Planning & Execution
Product Performance
Technical Maturity
Test and Evaluation
Program Requirements Factor -
Parameter Status
Resources Assessment ICPM versus PoPS
24
DA HIEF
YSTEMS NGINEER
OSD DAPSMission Capabilities / Requirements Generation Area
DoN POPSProgram Requirements Factor
Sub Area Factor Metric
CONOPs Mission Description CONOPsFoS / SoS Dependencies / Interfaces
Analysis of Alternatives
Validity and Currency Scope EvolutionLinkages and Traceability
Capabilities Reasonableness, Stability, and Testability Parameter StatusKPP / KSA
Requirements Assessment DAPS versus PoPS
25
DA HIEF
YSTEMS NGINEER
OSD DAPSResources Area
DoN POPSProgram Resources Factor
Sub Area Factor Metric
Budget Sufficiency & Phasing
Program Funding and Allocation Budget
Continuity and Stability Program Planning & Execution Factor -
Total Ownership Cost Estimating
Staffing Level Sufficiency of Numbers and Qualifications ManningContinuity and Stability
Program Schedule Overview
Viability Program Planning & Execution Factor -
Acquisition ManagementConstraints and Dependencies
Resources Assessment DAPS versus PoPS
26
DA HIEF
YSTEMS NGINEER
OSD DAPSManagement Area
DoN POPSProgram Planning & Execution Factor
Sub Area Factor Metric
Acquisition Strategy
Credibility Acquisition Management
Acceptability External Influences - Program Advocacy
External Influences - Fit in Vision
Knowledge-Based Decisions & Milestones
Statuary & Regulatory Compliance & Guidance
Entrance & Exit/Success Criteria
Certifications
Program & Project Management
Program/ Plan/Schedule Government Program Office PerformanceWBS
Management Structure & Communications
Management Methods, Metrics, and Techniques
Information Management (IDE, IT, Data Rights)
Management of Dependencies & External Interfaces (FoS/SoS))
External Influences - Interdependencies
Contracting Prime Contractor Management Contract Planning & ExecutionSubcontractor Management
Value Engineering Industry / Company Assessment
Management Assessment DAPS versus PoPS
Assessed at DoN Gate
Reviews and SETRs
27
DA HIEF
YSTEMS NGINEER
OSD DAPSTechnical Process Area
DoN POPSProgram Planning & Execution Factor
Sub Area Factor Metric
Design Considerations
System Assurance Technology Protection
Modular Open Systems Approach
Architecture
Environment, Safety, and Occupational Health
Spectrum Management
Sustainment as a Design Consideration
Corrosion
HSI
Requirements Development
Analysis and Decomposition
Management of Requirements
Technology and Maturity Integration Technical Maturity
Trade Studies and Approaches
Technical Baseline Technical Review Planning
Configuration Management
Baseline Stability
Technical Process Assessment DAPS versus PoPS
Assessed at DoN SETRs
Assessed at DoN Gate
Reviews and SETRs
28
DA HIEF
YSTEMS NGINEER
OSD DAPSTechnical Process Area
DoN POPSProgram Planning & Execution Factor
Sub Area Factor Metric
Engineering Tools Systems and Software Engineering (SSE) Tools
M&S Tools
Producibility and Production Planning Tools
Software Software Development Plan SoftwareEstimation
Design Verification
Test and Evaluation Plan Test and EvaluationVerification Correlation
Supportability Planning
Acquisition Logistics SustainmentPerformance-Based Logistics (PBL)
Sustainment
Technical Process Assessment DAPS versus PoPS, Con’t
Assessed at DoN Gate
Reviews and SETRs
29
DA HIEF
YSTEMS NGINEER
OSD DAPSProgram Performance Area
DoN POPSProgram Planning & Execution Factor
Sub Area Factor Metric
Effectiveness Design Capabilities Assessment Test and Evaluation
Technical MaturitySpecial Interest Areas - Readiness Level
Suitability Reliability Assessment SustainmentAvailability Assessment
Maintainability Assessment
Survivability Live Fire Test and Evaluation Assessment Test and Evaluation
Production Assessed Manufacturing Industry / Company AssessmentAssessed Quality
Program Performance Assessment DAPS versus PoPS
