Date post: | 12-Nov-2014 |
Category: |
Technology |
Upload: | nasapmc |
View: | 13,083 times |
Download: | 0 times |
Lessons Learned Implementing EVM in the EVA Office
Les Sorge/SGT, Inc.
Kristen C. Kehrer/NASA KSC
Project Management Challenge 2011
Long Beach, CA
February 9-10, 2011
Used with permission
2
Presentation Agenda
Background Lessons Learned
Management Reserve Budget versus Funding Risk Management Impact to EAC & MR Work Authorization Traceability & Scope Identical Structures and Titles for WBS and IMS WBS/Charge Numbers and EVM Control Accounts/Work Packages Institutional or Project OBS Data Integration with Multiple Tools Integration of Prime Contractor Data (1 control account or multiple) Setting and Validating the PMB
Questions & Answers Acronyms and Reference Documents
3
Background
Constellation (CxP) EVA Systems Project Office (ESPO) was selected as the first of two pilot projects for the NASA Earned Value Management (EVM) Capability Project The goal of the pilot project was to capitalize off of the EVM implementation efforts in
EVA to develop an Agency EVM capability, test the Agency EVM capability processes and tools for integrated in-house and contractor EVM, and to incorporate identified changes and lessons learned into the documentation for Agency wide streamlined EVM processes and training
The EVMS Capability team identified 33 gaps and 6 other areas of concern during the initial gap analysis of the ESPO pilot project in March, 2010
Closure plans were developed and documented with a plan to track and coordinate them to ensure proper vetting across affected organizations
The closure plans were to be implemented and tested through monthly reporting cycles – did the fix work?
The pilot project was prematurely terminated in June, 2010 because of changes/uncertainty within the Constellation Program and the ESPO
Lessons learned from the pilot project were documented as part of the EVM Capability Project Two issue papers were written and presented to the EVM Capability Project Steering Committee for
consideration Inconsistency Between Plan and Performance Business Rules and Actual Costs
4
Lessons Learned
5
Management Reserve
Driving Event CxP provided direction for no Management Reserve (MR) for the projects Misunderstanding of financial reserves (Unallocated Future Expense – UFE) vs budget reserves
(Management Reserve - MR) Lesson Learned – Allow projects to budget for and report MR to manage project
risks and in-scope changes and use the ANSI/EIA-748 standard term of MR
Graphic modified from Chambers/Gibby 07/2010
UFE
AgencyBaseline
Commitment
Management Baseline -
PROGRAM
CONGRESS & OMB
AUTHORIZED
UFE
AUTHORIZEDDistributed
Budget
UndistributedBudget
ManagementReserve
AUW
PMB
NASA EVM ProjectProgram Office
ProjectBudgetBase
UFE
Key: Held Back Funds/Funding Commitment Agreement Held Back Funds/Funding Management Agreement Not Strictly “Budget”, but should be inviolate Budget
Funding Process Budget Process
6
0
20
40
60
80
100
120
140
1Qtr
2QTR
3QTR
4QTR
1Qtr
2QTR
3QTR
4QTR
1Qtr
2QTR
3QTR
4QTR
1Qtr
2QTR
3QTR
4QTR
1Qtr
2QTR
3QTR
4QTR
GFY 2011 GFY 2012 GFY 2013 GFY 2014 GFY 2015
Budget versus Funding
Driving Event Misunderstanding the difference between budget and funding for EVM by all levels of
management No distinction between EVM budget and funding driving replans of the EVM Performance
Measurement Baseline (PMB) when funding changes occur EVM not fully integrated with current NASA financial processes and software tools
Lesson Learned – To support EVM, NASA culture must differentiate between budget and funding and not replan the PMB (eliminate variances and trends) for every funding change. The validity and accuracy of the performance measurement and forecast data rely on a stable PMB
6
Management Baseline - PROGRAM
UFE
AUTHORIZED
NASA EVM Project
Program Office
Annual PPBE (Funding Process)
EAC
The BCWS & BAC represent the work
The ACWP, ETC, & EAC represent the funds (i.e. money) required for that
work
Time Now
The BCWP represents the work accomplished
BAC
Initial Project Budget Base(BAC with MR = EAC)
Funding Plan Actuals/ETC/EAC
Funding LimitsObligations
VAC
DistributedBudget
UndistributedBudget
ManagementReserve
AUW
The funding limit (obligation) represents the maximum funds
available to accomplish the work
VAC > 15% Notification to Program / Mission Directorate
Define Project Budget (Budget Process)
7
Risk Management Impact to EAC & MR
Driving Event Project risks captured in Integrated Risk Management Assessment (IRMA) tool but no
process defined to utilize risks to develop initial project MR No process defined to include risks in an EAC update or how it relates to uncertainty
Lesson Learned – Projects should develop processes to tie risk assessments to the development and justification of MR and EAC updates
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2400 2450 2500 2550 2600
Pro
ba
bil
ity
70% Probability
70% Estimate
0.0000000
0.0050000
0.0100000
0.0150000
0.0200000
0.0250000
0.0300000
0.0350000
2250 2300 2350 2400 2450 2500 2550 2600
Probability Density Function Cumulative Distribution Function (S-Curve)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2400 2450 2500 2550 2600
Pro
ba
bil
ity
70% Probability
70% Estimate
0.0000000
0.0050000
0.0100000
0.0150000
0.0200000
0.0250000
0.0300000
0.0350000
2250 2300 2350 2400 2450 2500 2550 2600
Probability Density Function Cumulative Distribution Function (S-Curve)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2400 2450 2500 2550 2600P
rob
ab
ilit
y
70% Probability
70% Estimate
0.0000000
0.0050000
0.0100000
0.0150000
0.0200000
0.0250000
0.0300000
0.0350000
2250 2300 2350 2400 2450 2500 2550 2600
Probability Density Function Cumulative Distribution Function (S-Curve)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2400 2450 2500 2550 2600P
rob
ab
ilit
y
70% Probability
70% Estimate
0.0000000
0.0050000
0.0100000
0.0150000
0.0200000
0.0250000
0.0300000
0.0350000
2250 2300 2350 2400 2450 2500 2550 2600
Probability Density Function Cumulative Distribution Function (S-Curve)
Cost S-curve
Schedule S-curve
Joint Confidence Level (JCL)10 11 12
Risk No L X C Title Mar Apr MayJun Jul AugSep Oct NovDec Jan Feb Mar Apr MayJun Jul AugSep Oct NovDec Jan Feb
10147 3 x 3 CR 383 - Orion/EVA IRD rev. C
9966 4 x 4 [CSAFE] [EVA Suit] Occupant Protection Risk
= = = = = = >
1968 3 x 2 [EVA SE&I] System mass under-allocation
= = = = = = = = = = >
1575 3 x 3 [EVA Project] Software funding not adequate to meet core block-1 EVA and C3I lunar software requirements
>
1086 2 x 3 [EVA] Suit System IOC procurement reconciliation
= = = = >
5 3
43 8,9
2 5
1 6
1 2 3 4 5
Lik
elih
oo
d
Consequence
Risk Register
Uncertainty (Funding Process)Risk (Budget Process)
MR $ = [Risk Impact $ x Risk Factor] + rule of thumb for unknown risks
Sample Process1. Identify Initial Risks &
Risk Mitigation Plans2. Generate a risk factor
multiplier (0 to 1)3. Develop rule of thumb
for unknown risks4. Define MR $5. Calculate Uncertainty6. Update Risks & Risk
Mitigation Plans7. Update EAC with Risk
Updates8. Update JCL for KDP
gate reviews
Known Risks (known unknowns)
Unknown Risks (unknown unknowns)
???70% JCL = Prob [Cost] × Prob [Schedule | Cost]
Rules of thumb based on historical projects
8
Work Authorization Traceability & Scope
Driving Event The authorization document (PAD) did not authorize all of the ESPO work scope, i.e.
lunar content, even though work was being performed/actuals collected on that work ESPO had to maintain a separate Deltek Cobra project with this scope
Authorizing documents are not kept current with updates Limited scope description to reference for in-scope vs. out-of-scope discussions
Lesson Learned Create authorization document(s) that cover the entire scope of work and use
incremental authorization so all actual costs trace to an authorizing document Maintain all levels of authorizing documents to provide traceability Develop more detailed Statements of Work (SOW) between NASA Programs and
Projects
FAD - Formulation Authorization DocumentPCA - Program Commitment AgreementPAD - Project Authorization DocumentWAD - Work Authorization DocumentITA - Internal Task Agreement
* - Not currently considered an authorization document
NASA AA
MDAA
Program Mgr
Project Mgr
CAM
Center Org
Program FAD
Program PCA
Project FAD
PAD
WAD
JSCITA*
Funding Process Budget/Funding Process
9
Identical Structures and Titles for WBS and IMS
Driving Event The WBS element nomenclature and hierarchy is not consistently used within the
project software toolset Lesson Learned – Use only the project approved WBS numbering scheme
and nomenclature for integration and easy traceability of all project cost, schedule, and technical content If CAMs prefer the schedule outline to match the technical WBS (e.g. without the 6
digit project code and numbering hierarchy), then create a separate data field for the financial WBS
Structure Element Long TitleStructure Element Short Title
Project/WBSWBS Level
Performing Center
Exploration Systems Exploration Systems . .
Constellation Systems Constellation System . .
Constellation Systems Constellation System . .
Extravehicular Activity Extravehicular Activ 731384 .
Project Management Project Management 731384.01 WBS2 01
Systems Engineering Systems Engineering 731384.02 WBS2 01
NASA SE&I NASA SE&I 731384.02.31 WBS3 01
EVA SE&I Mgmt & Admin EVA SE&I Mgmt&Admin 731384.02.31.01 WBS4 01
EVA SEI System Engineering EVA SEI Sys Eng 731384.02.31.02 WBS4 01
EVA SEI Analyses EVA SEI Analyses 731384.02.31.03 WBS4 01
JSC-EVA SE&I EDAC1 Analysis JSC-EVA SEI EDAC1 731384.02.31.03.01 WBS5 72
JSC-EVA SE&I EDAC2 Analysis JSC-EVA SEI EDAC2 731384.02.31.03.02 WBS5 72
JSC-EVA Arch & Analysis IC PDR Tasks JSC-EVA Arch&Analys 731384.02.31.03.03 WBS5 72
EVA V&V IC PDR Tasks EVA V&V IC PDR Tasks 731384.02.31.04 WBS4 01
GRC-EVA PA&S GRC-EVA PA&S 731384.02.31.05 WBS4 22
JSC-CSSS SE&I JSC-CSSS SE&I 731384.02.32 WBS3 72
Deltek Cobra
Primavera
NASA MdM / SAP
10
WBS/Charge Numbers and EVM Control Accounts/Work Packages
Driving Event NASA accounting processes equate a charge number to a WBS number, therefore
new charge numbers essentially “change” the WBS NASA SAP accounting system rule that funding obligations must occur at the same
level the costs are collected, i.e. the charge number De-obligating funds from a charge number is a labor intensive process
Lesson Learned – Project EVM implementations plans need to define how best to work with the existing NASA processes and also meet the intent of general guidelines for WP
WBS Title Project/WBS
Extravehicular Activity 731384Project Management 731384.01
Project Management 731384.01.01
NASA Project Management 731384.01.01.31
JSC-EVA Project Mgmt thru KDP C 731384.01.01.31.01
JSC-External Review Supt thru KDP C 731384.01.01.31.02
GRC-EVA Project Mgmt thru KPD C 731384.01.01.31.03
LaRC-External Review Supt thru KDP C 731384.01.01.31.04
PP - JSC PM PDR to Orion2 731384.01.01.31.a
PP - GRC PM PDR to Orion2 731384.01.01.31.b
PP - JSC PM Orion2 to Orion10 731384.01.01.31.c
PP - GRC PM Orion2 to Orion10 731384.01.01.31.d
PP - JSC PM Orion12 to Orion18 731384.01.01.31.e
PP - GRC PM Orion12 to Orion18 731384.01.01.31.f
JSC-CSSS Project Managment 731384.01.01.32
Project Planning & Control 731384.01.02
NASA PP&C 731384.01.02.31
PP&C Mgmt 731384.01.02.31.01
JSC-PP&C Mgmt thru KDP C 731384.01.02.31.01.01
PP - JSC PPC Mgmt PDR thru CDR 731384.01.02.31.01.a
PP - JSC PPC Mgmt CDR thru Orion2 731384.01.02.31.01.b
PP - JSC PPC Mgmt Orion2 thru Orion18 731384.01.02.31.01.c
PP&C CM/DM 731384.01.02.31.02
Work Packages / Charge Codes
Planning Packages
Control AccountCxP Reporting Level
Control Account
WP Guidelines• WP should be segregated by
elements of cost• WP should have durations of four
months or less• During a replan, a WP should be
closed with s=p to date, with the remaining work planned in a new WP
11
Institutional or Project OBS
Driving Event The Organizational Breakdown Structure (OBS) for the project Responsibility Assignment
Matrix (RAM) was changed from the NASA center institutional organizations to the project organizational structure (i.e. integrated product teams)
NASA operates with two organizational breakdown structures as a system of checks and balances
Lesson Learned – The project needs to define the RAM OBS that best reflects the organizational authority and how work scope is subdivided between the orgs
Questions to ask: Who is ultimately responsible for the work
and has the budget authority? Who does the planning of the
schedule and budget for that scope of work?
To what level is control required or appropriate?
Do the institutional organizations just provide extra bodies or do they provide a “finished product”?
Should other NASA centers be listed as separate organizations and therefore, separate control accounts within the RAM?
Center Director
Directorate
DivisionBranch ManagerStaff
Staff
Staff
Branch ManagerStaff
Staff
Staff
Branch ManagerStaff
Staff
Staff
Institutional (functional) Org Structure
Staff members are grouped by functions or specialties such as Engineering, Safety & Mission Assurance, Public Affairs, etc.
Mission Directorate
Program ManagerProject
ManagerStaf
f
Staff
Staff
Project
ManagerStaf
f
Staff
Staff
Project
ManagerStaf
f
Staff
Staff
Project Coordination
Project Org Structure
Staff members are grouped into departments or product teams
Project Coordination
12
Data Integration with Multiple Tools
Driving Event The prime contractor uses different software tools (MS Project, Deltek MpM) then the
ESPO (Primavera, Deltek Cobra) Lesson Learned – Ideally, all project partners should use the same
software toolset, however, this may not be practical, so request for proposals (RFP) should contain specific language in the Data Requirements Document(s) (DRD) for EVM software tool interfaces The project should develop an integration methodology prior to the RFP to help
define the interface requirements
MS Project
Deltek MpM
DeltekwInsight
DeltekCobra
DeltekwInsight
Primavera
• Standard Code Fields• Unique Task IDs• IMP Codes• WBS• EVM Percent Complete
• File Formats• File Types (.xls, .xml, etc)• Etc.
Interface Requirements
Prime Contractor EVA Systems Project Office
13
Integration of Prime Contractor Data(1 control account or multiple)
Driving Event ESPO defined multiple control accounts (and therefore multiple WBS/charge
numbers) for the prime contractor Each charge number required obligations which sometimes created “cost over obligations”
on a charge number even though the total cost was within the total obligations
Lesson Learned – Projects should develop contractor data integration plans and weigh the pros/cons of the different options
Single Control Account Multiple Control AccountsEasier to Obligate costs for a single WBS/CA in SAP
Easier to integrate and analyze EV data at an IPT level
Easier to integrate Contractor EVM data into the Project data with only one CA
NASA CAMs own their associated Contractor CA and work scope vs a single CAM for the entire contractor
Less maintenance/booking of contractor budgets in project PMB (contractor UB, MR, etc.)
All (NASA and contractor) functional work captured in the same WBS, i.e. all Mgmt in 1.0, etc.
Less impact if the project WBS or contractor WBS changes since no need to keep the them in-sync
Reinforces the “one project team” concept (NASA and prime)
731384.06.04.01Suit Element
.31 Govt CA
.32 Prime CA
WP / PP
WP / PP
731384.06.04.03Vehicle Interface Element
731384.06.04EVA HW Development
.31 Govt CA
.32 Prime CA
WP / PP
WP / PP
Multiple Control Accounts
731384.06.04.01Suit Element
Govt CA
WP / PP
731384.06.04.03Vehicle Interface Element
731384.06.04EVA HW Development
Govt CA
WP / PP
Example of Single Control Account
731384.06.04.05CSSS Contract
(Prime CA)
Pros / Cons
14
Setting and Validating the PMB
Driving Event Prime contractor operated for a year under letter contract without a Performance
Measurement Baseline (PMB) Constant, significant changes to the Constellation Program/Projects funding did not
allow for a stable baseline so as to set a PMB A PMB was developed for an Integrated Baseline Review (IBR), but the announcement to
cancel the Constellation Program and associated reduction in funding changed the focus of the IBR to an FY10 review
Prolonged proposal submittal and authorization process encumbered the change process
Lesson Learned Clearly define expectations and requirements to initialize the PMB and begin EVM
reporting to include letter contracts Utilize existing NASA resources such as the IBR Handbook, the EVM Handbook, and
the Schedule Handbook to define appropriate EVM requirements Projects and contractors should strive for timely contract authorization and
definitization of the base contract and subsequent contract modifications Given the fluid nature of funding, develop “right-sized” products
Schedules and budgets that are not overwhelming to update Use of planning packages and summary level planning packages Utilization of rolling wave planning
15
Questions & Answers
16
Acronyms and Reference Documents Acronyms
AUW – Authorized Unpriced Work CA – Control Account CAM – Control Account Manager CPR – Cost Performance Report CSSS – Constellation Space Suit System CxP – Constellation Program DRD – Data Requirements Document EAC – Estimate at Completion ESPO – EVA Systems Project Office ETC – Estimate to Completion EVA – Extravehicular Activity EVM – Earned Value Management EVMS – Earned Value Management System IBR – Integrated Baseline Review IMS – Integrated Master Schedule IPT – Integrated Product Team IRMA – Integrated Risk Management Application ITA - Internal Task Agreement JCL - Joint Confidence Level JSC – Johnson Space Center MR – Management Reserve OBS – Organizational Breakdown Structure PAD – Project Authorization Document PMB – Performance Measurement Baseline PP – Planning Package PPBE - Planning, Programming, Budgeting, and Execution RAM – Responsibility Assignment Matrix SAP – Systems Application and Products UFE - Unallocated Future Expense WBS – Work Breakdown Structure WP – Work Package
Reference Documents NPR 7120.5D – NASA Program and Project
Management Processes and Requirements CxP 72179 Anx01 – CxP ESPO EVM
Implementation Plan ANSI/EIA-748 – Earned Value Management
Systems