Cpm 200 C technical performance measures ipm2016

CPMIPM2016RightsReservedbytheOrganizationontheCoverPage

IPM2016

CPM-200: Scope and Organization Management

Lesson C: Using Technical Performance Measures to Inform Earned Value Performance

Glen B. Alleman Thomas J. CoonceNiwot Ridge, LLC Institute for Defense [email protected] [email protected]+1 303 241 9633 +1 703 362 2568


Learning Objectives

• TLO #1: The student will recognize what a Technical Performance Measure (TPM) is in the context of the Professional Education Program Case Study as well as DOD and INCOSE Systems Engineering Guidance.

• TLO #2: The student will be able identify and select TPMs using the example Work Breakdown Structure from the Case Study.

• TLO #3: The student will learn how to prepare descriptions of TPMs needed to assess the physical “progress to plan” of the program.

• TLO #4: The student will able to judge the validity of a TPM from a list of TPM’s from the notional program.

2


Components we’ll meet along the way to a credible PMB

ObjectiveStatusandEssentialViewstosupporttheproactivemanagementprocessesneededtokeeptheprogramGREEN

RiskManagement

SOWSOO

ConOpsWBS TechnicalandOperational

Requirements

CWBS&CWBSDictionary

IntegratedMasterPlan(IMP)

IntegratedMasterSchedule(IMS)

MeasuresofEffectiveness

MeasuresofPerformance

MeasuresofProgress

KeyPerformanceParameters

ProgramSpecificKeyPerformanceParameters

TechnicalPerformanceMeasures

EarnedValueManagementSystem

TPMsLiveHere

PerformanceMeasurementBaseline

TLO#1

3


Can Earned Value Alone Get Us To Our Destination?

• How do we increase visibility into program performance?• How do we reduce cycle time to deliver the product?• How do we foster accountability?• How do we reduce risk?• How do we start our journey to success?

IncreasingtheProbabilityofSuccessmeans…ConnectingTheDotsBetweenEVMandTPMtoReachOurDestination

TLO#1

4


Increasing the Probability of Program Success Means …

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB

Building A Credible Performance Measurement Baseline

This is actually harder than it looks!

TLO#1

5


Identifying the TPMs starts with good Systems Engineering

TLO#1

6


Some More Guidance

Systemsengineeringusestechnicalperformancemeasurements tobalancecost,schedule,andperformancethroughoutthelifecycle.Technicalperformancemeasurementscompareactualversusplannedtechnicaldevelopmentanddesign.Theyalsoreportthedegreetowhichsystemrequirementsaremetintermsofperformance,cost,schedule,and progressinimplementingrisk handling.Performancemetricsaretraceabletouser–definedcapabilities.―DefenseAcquisitionGuide

(https://dag.dau.mil/Pages/Default.aspx)

In The End ― It’s Always About Systems Engineering

TLO#1

7


This Has All Been Said Before

… the basic tenets of the process are the need forseamless management tools, that support anintegrated approach … and “proactive identificationand management of risk” for critical cost, schedule,and technical performance parameters.― Secretary of Defense, Perry memo, May 1995

Why Is This Hard To Understand?§ We seem to be focused on EV reporting, not the use of

EV to manage the program.§ We think getting the IPMR out the door is the end of

Program Planning and Control’s efforts, not the beginning.

TPM Handbook 1984

TLO #1

8


Back To Our Technical Performance Measures

Technical Performance Measures do what they say,

Measure the Technical Performanceof the product or service produced by

the program.

TLO#2

9


To develop TPMs we first need to define some other measures

• Measures of Effectiveness (MoE)• Measures of Performance (MoP)• Key Performance Parameters (KPP)• Then we can define the Technical

Performance Measures (TPM)

10


Measure of Effectiveness (MoE)

Measures of Effectiveness …• Are stated in units meaningful to the buyer,• Focus on capabilities independent of any technical

implementation,• Are connected to the mission success.

Operational measures of success that are closely related to the achievements of the mission or operational objectives

evaluated in the operational environment, under a specific set of conditions.

MoE’s Belong to the End User

TLO#2

“TechnicalMeasurement,”INCOSE–TP–2003–020–01

11


Measure of Performance (MoP)

Measures of Performance are …• Attributes that assure the system has the

capability and capacity to perform,• Assessment of the system to assure it meets

design requirements to satisfy the MoE.

Measures that characterize physical or functional attributes relating to the system operation, measured or estimated

under specific conditions.


MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C

TLO#2

12


Key Performance Parameters (KPP)

Key Performance Parameters …• Have a threshold or objective value,• Characterize the major drivers of performance,• Are considered Critical to Customer (CTC).

Measures that Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program

The acquirer defines the KPPs during the operational concept development – KPPs say what DONE looks like

TLO#2


13


Technical Performance Measures (TPM)

Technical Performance Measures …• Assess design progress,• Define compliance to performance requirements,• Identify technical risk,• Are limited to critical thresholds,• Include projected performance.

Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical

requirement or goal

TLO#2


14


TPMs start with the WBSTLO#2

1.0SaferCounties/Safer

Cities

1.1SC2PrimeMission

Equipment

1.1.1.CameraSystems

1.1.1.1Camera

1.1.1.2CameraInterfaceHardware

1.1.1.3CameraIntegration

1.1.2WirelessTowerSystem

1.1.2.1WirelessTower

1.1.2.2TowerInterfaceTower

1.1.2.3TowerIntegration,Test,Assembly,and

Checkout

1.1.3MonitoringStations

1.1.3.1PCStations

1.1.3.2PCInterfaceHardware

1.1.3.3MonitoringStationsIntegration,Test,Assemblyand

Checkout

1.2SystemEngineering

/ProgramManagement

1.3SystemTestand

Evaluation

1.4Data

1.5Training

Let’s focus on the wireless towers and the services they provide to public safety, as well as convenience for the citizens.We’ll need MOPs, MOEs, KPPs, before we define the TPMsThen we’ll connect these TPM’s to assessing the performance of the work in the IMS.

15


§ This is where TPMs are connected with the MoE’s and MoP’s

§ For each deliverable from the program, all the “measures” must be defined in units meaningful to the decision makers.

§ Here’s some “real” examples.

Connecting the MoE, MoP, KPP, and TPMs

1. Providebandwidthfor5,000simultaneousvoiceanddatausers

2. Providelocationservicefor911

3. Provide99.5%availabilitytousers

MeasuresofEffectiveness(MoE)

1. NetReady2. E911LocationQuality3. Phone

Interoperability4. Manpower5. Availability

KeyPerformanceParameters(KPP)

1. NetReady§ IPv4/6compliance§ 1GbEthernet

2. Locationquality§ Accuracythreshold@60M

§ Integritythreshold@40M

3. Phoneinteroperability§ Processingcapabilitymeetsgrowthmatrix

4. Manpower§MTBC>1000hrs§MCM<2hrs

5. Availability§ Clearthreshold>99%§ Noisethreshold>90%

MeasuresofPerformance(MoP)

1. NetReady§ Standardmessagepackets

2. LocationQuality§Multipathallocationbudget

§Multipathbiasprotection3. PhoneInteroperability

§MOSAcompliant§ Civilcompliant

4. Manpower§ Operatingelapsedtimemeters

§ Standbyelapsedtimeindicators

5. Availability§ Phasecentervariations

TechnicalPerformanceMeasures(TPM)

MissionCapabilitiesandOperationalNeed

TechnicalInsight– Riskadjustedperformancetoplan

TLO#2

16



Identifying Technical Performance Measures

Technical Performance Measures must be technicaland must describe the performance in units of measure

meaningful to the decision makers.TPM’s are not counts of things delivered. TPMs are not assessments of completed work through effort or cost.

TPMs are measures of Technical Performance

TLO#3

17


How can we possibly know we’re on schedule or on budget without knowing if we’re technically compliant?

TLO#3

18


Measures of Technical Parameters†

Attribute Description

AchievedtoDate § Measuredtechnicalprogressorestimateofprogressinunits ofperformance

CurrentEstimate § Valueofatechnicalparameterthatispredictedtobeachievednowor inthefuture

Milestone § Pointintimewhen anevaluationofthetechnicalmeasureisaccomplished

PlannedValue § PredictedvalueofthetechnicalparameterPlannedPerformance

Profile§ Profilerepresentingtheprojecttimephased

demonstrationofatechnicalparameterToleranceBand § Managementalertlimits

Threshold § Limiting acceptablevalueofatechnicalparameter

Variances § Demonstratedtechnicalvariance§ Predictedtechnicalvariance

TLO#3

†INCOSESystemsEngineeringHandbook

19


A Familiar Graphic for TPMs

Variance

PlannedValue

PlannedProfileCurrentEstimate

Milestones

Threshold

UpperControlLimit

LowerControlLimit

MeanToBetweenFailure

Time=ProgramMaturity

AchievedtoDate

TPM

TLO#3

20


What Do We Need To Know About This Program Through The TPMs?

• What WBS elements represent the TPMs?• What Work Packages produce these WBS

elements?• Where do these Work Packages live in the IMS?• What are the Earned Value baseline values for

these Work Packages?• How are we going to measure all these variables?• What does the curve look like for these

measurements?

TLO#3

21


Let’s Connect Some More DotsTLO#3

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB

❺DeliverablesdefinedintheSOW,tracedtotheWBS,withnarrativesandMeasuresofPerformance(MoP)

❹BudgetattheWorkPackagelevel,rolledtotheControlAccountsshowingcostspreadsforallworkintheIMS

❻MeasuresofPerformance(MOP)foreachcriticaldeliverableintheWBSandidentifiedineachWorkPackageintheIMS,usedtoassessmaturityintheIMP

❶TheProductsandProcessesina“wellstructured”decomposition,traceabletothedeliverables

❷ IMScontainsalltheWorkPackages,BCWS,Riskmitigationplans,withtracestotheIMPmeasuringincreasingmaturitythroughMeasuresofEffectiveness(MOE)andKPPs

❸TechnicalandProgrammaticRisks ConnectedthroughtheWBS,RiskRegister,IMPandIMS

ThePMBistheDocumentofRecordfortheProgramPerformanceisMeasured

throughthePMB

22


Raison d'etre for Technical Performance Measures

The real purpose of Technical Performance Measures is to reduce Programmatic and Technical RISK

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB

TLO#4

23


Key Elements of A Good TPM

• Traceability – Requirements to WBS to TPMs to EV control accounts.

• Impact – How much WBS work, & therefore EV money, is covered by the TPM(s)? What is effect?

• TPM Banding/Sensitivity – What banding (R/Y/G) and sensitivity (EV impact) should be used for each TPM?

• Technical Readiness Level – What’s the state of the technology supporting the requirement(s) for which TPM is a metric?

ImplementingTechnicalPerformanceMeasurement,MikeFerraro,GeneralEngineer,PEO/SYSCOMConference

TLO#4

24


Determination of Physical % Complete must be informed by both task completion and technical

status

4Progress of a set of tasks – Drawings Completed– Lines of Code Written– Work Products Produced– Reviews Completed

• Progress of the technical status – Critical TPM Achievement– System Capabilities Met– Quality of Work Products– System Under Review Acceptable

Measuringprogressofwhatpeopledo

MeasuringprogressoftheResultofwhatpeopledo

Progressismeasuredbyeffectiveness ofoutcomestotheenduser

Physical % Complete P%C

TLO#4

BCWPii=1

N

∑ = BCWSi × P%Cii=1

N

∑

25


Connecting the EV VariablesIntegratingCost,Schedulele,andTechnicalPerformance

AssuresProgramManagementhastheneededperformanceinformationtodeliveron‒time,on‒budget,andon‒specification

TechnicalPerformanceMeasuresCost Schedule

ConventionalEarnedValue

+=

§ MasterScheduleisusedtoderiveBasisofEstimate(BOE)nottheotherwayaround.

§ Probabilisticcostestimatingusespastperformanceandcostriskmodeling.

§ Labor,Materiel,andotherdirectcostsaccountedforinWorkPackages.

§ Riskadjustmentsforallelementsofcost.

CostBaseline§ EarnedValueisdilutedbymissingtechnicalperformance.

§ EarnedValueisdilutedbypostponedfeatures.

§ EarnedValueisdilutedbynoncompliantquality.

§ AllthesedilutionsrequireadjustmentstotheEstimateatComplete(EAC)andtheToCompletePerformanceIndex(TCPI).

TechnicalPerformance§ Requirementsaredecomposedintophysicaldeliverables.

§ DeliverablesareproducedthroughWorkPackages.

§ WorkPackagesareassignedtoaccountablemanager.

§ WorkPackagesaresequencedtoformthehighestvaluestreamwiththelowesttechnicalandprogrammaticrisk.

ScheduleBaseline

TLO#4

26


TPM ChecklistMoE MoP TPM

Traceabletoneeds,goals,objectives,andrisks.

TraceabletoapplicableMOEs,KPPs,systemlevelperformancerequirements,andrisks.

TraceabletoapplicableMoPs,systemelementperformance,requirements,objectives,risks,andWBSelements.

Definedwith associatedKPPs.

Focusedontechnicalrisksandsupportstrades betweenalternativesolutions.

Furtherdecomposed,budgeted,andallocatedtolowerlevelsystemelementsintheWBSandIMS.

EachMoEindependentfromothers.

Providedinsightintosystemperformance.

Assignedanowner,theCAMandWorkPackageManager.

EachMoEindependentoftechnicalanysolution.

Decomposed, budgetedandallocatedtosystemelements.

Sourcesofmeasureidentified andprocessesforgeneratingthemeasuresdefined.

Address the requiredKPPs. Assignedan“owner,”theCAMandTechnical Manager.

Integratedintotheprogram’sIMSas partoftheexitcriteriafortheWorkPackage.

TLO#4

27


Summary of TPM Benefits

• Measures of results based on technical achievement

• Integrated technical performance with budget and schedule – providing an integrated systems approach to performance measurement

• Establishes the linkage between engineering and cost/schedule considerations in program management decision making

• Measures the programmatic impact of technical variance in terms of risk

• Reduces subjectivity through the establishment of technical performance baseline

TLO#4

28


Past Issues with Earned Value Management and Scheduling

• Ability to take credit for all work accomplished• How to defend what work was earned• Identifying EV milestones in the schedule focused

attention on those tasks so that maximum earned value could be taken

• IMS was too detailed to manage the program• CAM’s used other means to manage and assess

progress related to their work; i.e. detailed schedules• Detailed schedules did not reflect what was reported in

IMS

29

Solution(– Quantifiable(Backup(Data((QBD)


Quantifiable Backup Data (QBD)

• A QBD is a supplement to the IMS to help manage discrete work at a lower level than is maintained in the IMS

• The QBD measures the work objectively– 748-C calls out on Page 1– Objectively assess accomplishments at the work

performance level• QBD’s are part of the PMB• QBD’s do not replace the EVMS Earned

Value Techniques (EVT)30


Quantifiable Backup Data (QBD)

• A detailed listing of tasks necessary to complete all scope in a work package during the defined period of performance.

• It is an approach used to objectively measure performance– Each task on the list is weighted – total weighting equals 100% of the

work package BAC (weighting is important, should not be equal weighting on every task).

– Upon completion of QBD development the QBD is placed under configuration control.

– The CAM assesses physical percent complete of each QBD task.– The percent complete is calculated from the cumulative assessments.

• The purpose of the QBD is to:– Ensure and demonstrate that all contract work is accounted for;– Ensure the schedule and budget are realistic and achievable;– Mitigate schedule and budget risks; and,– Provide a basis for objectively assessing progress for discretely

measured work packages.

31


32



What We’ll Do In Our Hands On Work Shop

Based on a notional case study for a county surveillance system, we will strategize about the system MOEs

and MOPs and decide upon appropriate TPMs to inform our

program performance and BCWP

33


Case Study Set Up

• The county is concerned about controlling disturbances, mitigating crime, and having forensic video and sound evidence for legal purposes

• The county desires to procure a surveillance system with cameras and sound devices at strategic locations, send the data feeds via wireless technology, monitor, store and retrieve selected incidents

34


Case Study Set Up (Continued)

• The county wants near “real time” response and desires video quality such that they can distinguish facial features and read a license plates within 20 feet from the camera

• It also wants the system to alert operators when there is evidence of a disturbance

• It wants equipment that will allow them to store five years worth of data feeds collected 24/7

35


Case Study Set Up (Concluded)

• The county has submitted a Request for Proposal (RFP) to procure, install, test and train personnel. The county has set aside of budget of $40M and wants the system operational within 18 months. Because of the nature of the procurement, the county has required monthly IPMRs (See Handout 1 for details)

• You are the proposal manager for the contract. Your immediate job is to – Determine the system’s “Measure of Effectiveness”– Select appropriate Technical Performance Measures to

use to inform or drive reported cost and schedule progress– Propose how you will ensure the county that cost and

schedule progress is consistent with measured technical progress

36



Handout # 1

Safer County/Safer City Surveillance Background and

Technical Requirements

37


Exercise # 1

• What are possible Measures of Effectiveness (MOEs) of the System?

• What are possible Measure of Performance (MOPs) of the System?

• What are possible Technical Performance Measures (TPM) that could be used to drive or inform the cost and schedule progress?

38


AllthesemeasuresusedtoinformBCWP

Reminder: MOEs, MOPs, and TPMs are Connected to the IMP/IMS

SOWConOps

WBSCWBS

TechnicalPerformanceMeasures

MessagesofPerformance

MeasuresofEffectivenessKPPs

IntegratedMasterPlan

IntegratedMasterSchedule

PerformanceMeasurement

Baseline

TechnicalandOperationalRequirements

39



Handout # 2

Safer County/Safer City Surveillance System

School Solution

40


Exercise # 2

• Discuss school solution

41


Exercise # 3

• Which WBS elements & IMS Work Packages need to be informed by TPM progress?

42



Handout # 3

Safer County/Safer City Surveillance System

Work Breakdown Structure and Integrated Master Schedule

43



Presentation of Workshop Results

With our MOE’s, MOP’s, KPP’s, and TPM’s let’s assess out needed

capabilities against the technical solution

44


Questions???

45


OFFICIAL VERSION

COLLEGE OF PERFORMANCE MANAGEMENTPROGRAM OF INSTRUCTION

500C VERSION 1.225 Mar 2015

Approvals:

William Mathis, V.P of Education and Certification 25 Mar 2015 [Add appropriate approval individual name and title] Date

[Add appropriate approval individual name and title] Date

[Add appropriate approval individual name and title] Date

46


RECORD OF CHANGES

VERSION NUMBER

DATE NUMBER OF FIGURE, TABLE OR PARAGRAPH

A*M D

TITLE OR BRIEF DESCRIPTION CHANGEREQUESTNUMBER

1 22 Feb 2014 A,M Format change for IPM 20141.1 18 Sep 2014 A,M Format change for IPMW 20141.2 25 Mar 2015 A,M Format changes

Record of Changes

47

Date post:	11-Jan-2017
Category:	Technology
Upload:	glen-alleman
View:	189 times
Download:	3 times

Cpm 200 C technical performance measures ipm2016

Technology