Program Management Program Management with Requirements ...

11

Program Management Program Management with Requirementswith Requirements--

Based Measurement Based Measurement IFPUG Fall Conference 2007IFPUG Fall Conference 2007

Terry VogtBooz Allen Hamilton

8283 Greensboro DriveMcLean, VA 22102

22

Program Management with Program Management with RequirementsRequirements--Based MeasurementBased Measurement

Agenda

Rationale: The sizes of contemporary major software projects Rationale: The sizes of contemporary major software projects and development programs are becoming increasingly huge. and development programs are becoming increasingly huge. They are difficult to manage, and hold opportunity for enormous They are difficult to manage, and hold opportunity for enormous failure. Conversely, they provide the potential for massive failure. Conversely, they provide the potential for massive savings & success.savings & success.

• Essentials of Requirements-Based Software Measurement

• Application and Benefits• Improving Measurement Quality • Dealing with Change and Uncertainty

33


Essential Factors in RequirementsEssential Factors in Requirements--Based Based Software MeasurementSoftware Measurement

•• Key role of functional requirements Key role of functional requirements

•• Other significant measurement factorsOther significant measurement factors

•• Use of Function PointsUse of Function Points

•• Use of estimation toolUse of estimation tool

44


•• Key role of functional requirements Key role of functional requirements Identify what is to be providedIdentify what is to be provided

Basis for contractual obligationBasis for contractual obligation

Apply to new development, enhancements, packagesApply to new development, enhancements, packages

Key to sizing workKey to sizing work

Key to evaluating package capabilityKey to evaluating package capability

Fundamental to user satisfactionFundamental to user satisfaction

Basis for reusability considerationsBasis for reusability considerations

55


•• Other significant measurement factorsOther significant measurement factorsFor project managementFor project management

For program managementFor program management

For organizational improvementFor organizational improvement

For customer satisfactionFor customer satisfaction

For budgetingFor budgeting

For competitive successFor competitive success

1 2 3 4 5 6 7 8 9 10 11

4

8

12

16

20

Months

Est. Schedule

Defects Inserted

Defects Rem oved

Potential Defects

Delivered Defects

1 2 3 4 5 6 7 8 9 10 1 1 12 13 1 4

4 0

8 0

120

160

200

1% 10% 20% 30% 40% 50% 60% 70% 80% 90% 99%0

4

8

12

16

20

10%: Best

50%: Median

90%: Worst

66


Function points measure software size based on the Function points measure software size based on the functionality requested by and provided to the end userfunctionality requested by and provided to the end user

Function point counting resources

User/analyst interviewsRequirements documentsDesign documentsData dictionariesUse casesUser guidesScreen capturesActual softwareEntity-relationship modelsSemantic object models

Function point counting Function point counting resourcesresources

User/analyst User/analyst interviewsinterviewsRequirements Requirements documentsdocumentsDesign documentsDesign documentsData dictionariesData dictionariesUse casesUse casesUser guidesUser guidesScreen capturesScreen capturesActual softwareActual softwareEntityEntity--relationship relationship modelsmodelsSemantic object Semantic object modelsmodelsFunction points represent Function points represent logicallogical size, as size, as

opposed to opposed to physicalphysical size (like SLOC or objects)size (like SLOC or objects)

77


•• Use of Function PointsUse of Function PointsRequirements managementRequirements management

Package evaluationPackage evaluation

EstimationEstimation

Change managementChange management

EVMEVM

Process improvementProcess improvement

Performance evaluationPerformance evaluation

Program managementProgram management

88


•• Requirements Source MaterialRequirements Source MaterialMany available sources: RFPs / RFIs, Proposals, user Many available sources: RFPs / RFIs, Proposals, user documentation, Use Cases, BPM in ERP software documentation, Use Cases, BPM in ERP software development, discussions & papers, etc. development, discussions & papers, etc.

Requirements translated into function points for sizeRequirements translated into function points for size

•• Use of software development estimation toolUse of software development estimation toolUse of historic experience databaseUse of historic experience database

Qualitative performance capabilities are criticalQualitative performance capabilities are critical

Use of output reports and comparative analysisUse of output reports and comparative analysis

May be able to equate requirements and SLOCMay be able to equate requirements and SLOC

99


Example: SEERExample: SEER--SEMSEM

Parametric software Parametric software development estimation tooldevelopment estimation tool

Model general project Model general project characteristicscharacteristics•• PlatformPlatform•• ApplicationApplication•• Acquisition methodAcquisition method•• Development methodDevelopment method•• Development standardDevelopment standard

Adjust input parameters to Adjust input parameters to reflect actual development reflect actual development environmentenvironment•• Personnel capabilitiesPersonnel capabilities•• Development Development

environmentenvironment•• Labor ratesLabor rates•• Product requirementsProduct requirements

Calibrate to historical data

Tool produces a range of possible Tool produces a range of possible cost, effort, and schedule values cost, effort, and schedule values with corresponding certainties, with corresponding certainties, as well as defect estimates and as well as defect estimates and other metricsother metricsCalibrate to historical data

1010


Use of estimation tool Use of estimation tool -- ExampleExample

A framework with which proposals can be comparedA framework with which proposals can be comparedComparing certainty levels across Comparing certainty levels across the estimate continuum is key to the estimate continuum is key to understanding relative value or understanding relative value or risk of any given proposal risk of any given proposal

Cost Range

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

1% 10% 20% 30% 40% 50% 60% 70% 80% 90% 99%

Certainty Level

Cos

t ($

Mill

ions

)

Proposal B

Proposal A

Proposal C

Schedule Range

0.0

10.0

20.0

30.0

40.0

50.0

60.0

1% 10% 20% 30% 40% 50% 60% 70% 80% 90% 99%

Certainty Level

Dur

atio

n (M

onth

s)

Proposal B

Proposal A

Proposal C

Review estimates in this context Review estimates in this context to identify any idiosyncrasies or to identify any idiosyncrasies or inconsistenciesinconsistencies

1111


Application and Benefits of Application and Benefits of requirementsrequirements--based software based software measurementmeasurement

•• Software developmentSoftware development

•• Vendor package evaluationVendor package evaluation

•• Analysis of alternativesAnalysis of alternatives

•• Project and program managementProject and program management

1212

Application and Benefits of RequirementsApplication and Benefits of Requirements--Based Software MeasurementBased Software Measurement

•• Software developmentSoftware development

Correspondence between whatCorrespondence between what’’s required and whats required and what’’s s delivereddelivered

•• EVM, testing, user acceptanceEVM, testing, user acceptance

Clarity of the impact of change requestsClarity of the impact of change requests•• Effect on effort, cost, scheduleEffect on effort, cost, schedule

Usage as negotiation tool between customer and Usage as negotiation tool between customer and project teamproject team

•• Cost & schedule control with user buyCost & schedule control with user buy--inin

Means of identifying opportunities for reuseMeans of identifying opportunities for reuse•• Opportunistic reuse, planned reuse, product lines Opportunistic reuse, planned reuse, product lines

1313

Size estimates can be directly linked Size estimates can be directly linked to baseline requirementsto baseline requirements

Discrete requirements statements and function points are Discrete requirements statements and function points are linked in a project plan and set up an EVM framework linked in a project plan and set up an EVM framework

ID Task Name Duration Budget1 Design 24 days $3,375.002 Code 15 days $4,098.003 Test 18 days $6,027.00

W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12Month 1 Month 2 Month 3

ID Task Name Duration Budget1 Design 24 days $3,375.002 Code 15 days $4,098.003 Test 18 days $6,027.00

W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12Month 1 Month 2 Month 3

Planned work allocated to each requirement

Planned schedule with phase milestones

Implications for correspondence, clarity and negotiationImplications for correspondence, clarity and negotiation

1414


•• Vendor package evaluationVendor package evaluationAssessment of package capabilityAssessment of package capability

Gap analysis: base package versus requirementsGap analysis: base package versus requirements

Estimation of effort needed to fulfill requirements Estimation of effort needed to fulfill requirements

Useful size of package evaluated for normalized valueUseful size of package evaluated for normalized value

Package fitness to purpose:Package fitness to purpose:

Vendor gap fit analysis on Vendor gap fit analysis on each user functional each user functional requirementrequirement

Gaps are identified by degree: Gaps are identified by degree: 3 = none, 2 = minor, 1 = 3 = none, 2 = minor, 1 = major, 0 = totalmajor, 0 = total

Applied as weights to effortApplied as weights to effort

1515


•• Analysis of alternativesAnalysis of alternativesBuild versus buy comparisonBuild versus buy comparison

Package comparisonPackage comparison

Acquisition alternativesAcquisition alternatives•• Function points estimated from requirementsFunction points estimated from requirements

•• Package size adjusted for requirements fitPackage size adjusted for requirements fit

•• Requirements not fully satisfied by package become factor for Requirements not fully satisfied by package become factor for additional development costadditional development cost

1616


•• Analysis of alternatives Analysis of alternatives Acquisition alternatives Acquisition alternatives -- continuedcontinued

•• Identification of package fixed costsIdentification of package fixed costs

•• Combine package costs with additional development Combine package costs with additional development cost required to fully satisfy user requirementscost required to fully satisfy user requirements

1717


•• Project and program managementProject and program managementManaging to budget capsManaging to budget caps

Requirements scheduling for release managementRequirements scheduling for release management

Acquisition strategy comparisons Acquisition strategy comparisons

1818


Acquisition strategy comparisons Acquisition strategy comparisons –– example: Reuseexample: Reuse

1919


Improving Measurement Quality Improving Measurement Quality

•• Reducing uncertaintyReducing uncertainty

•• Requirements traceabilityRequirements traceability

•• Performance feedbackPerformance feedback

2020

Improving Measurement QualityImproving Measurement Quality

•• Reducing uncertaintyReducing uncertaintyRequirements uncertainty affects project planningRequirements uncertainty affects project planning

•• Uncertainty increases the need to raise estimated size Uncertainty increases the need to raise estimated size and corresponding effort and costand corresponding effort and cost

•• Uncertainty expands ranges necessary to express Uncertainty expands ranges necessary to express potential outcome potential outcome ––> risk feeds on itself> risk feeds on itself

What decreases uncertaintyWhat decreases uncertainty•• User inputUser input

•• Project structureProject structure

•• Progress on a project, experience and reliable historyProgress on a project, experience and reliable history

•• Stability in environment: technical, managerial, userStability in environment: technical, managerial, user

Uncertainty may decrease with time, or may notUncertainty may decrease with time, or may not•• Time yields experienceTime yields experience

•• Speed can reduce exposure to changeSpeed can reduce exposure to change

2121


•• Requirements traceabilityRequirements traceabilityBest practices Best practices

•• Enables requirements fulfillmentEnables requirements fulfillment

•• Part of optimal configuration managementPart of optimal configuration management

•• Essential for impact analysisEssential for impact analysis

Allows association of size with specific componentsAllows association of size with specific components•• Ensures component and total size measurements can Ensures component and total size measurements can

be maintained over the life of the project & applicationbe maintained over the life of the project & application

Enables precise adaptive measurementEnables precise adaptive measurement•• Allows measurements to change with requirementsAllows measurements to change with requirements

•• Enables reEnables re--estimation and estimation and ““what ifwhat if”” explorationexploration

•• Supports maintenance measurementSupports maintenance measurement

•• Supports portfolio management through performance Supports portfolio management through performance evaluationevaluation

2222


•• Performance feedback of actual results to Performance feedback of actual results to measurement and estimation processesmeasurement and estimation processes

Actuals provide data to evaluate estimation processActuals provide data to evaluate estimation process•• Accurate records of staffing, effort, duration, expenseAccurate records of staffing, effort, duration, expense

•• Calibration of estimates to actual local performanceCalibration of estimates to actual local performance

Organizational info provides Organizational info provides ““opportunity dataopportunity data””

-- Keys to learning, predicting, schedulingKeys to learning, predicting, scheduling•• Utilization dataUtilization data

Delays in acquiring requested resourcesDelays in acquiring requested resources

Potential overstaffing leading to Potential overstaffing leading to ““paddingpadding””

•• Capability dataCapability data

Numbers of specific resource typesNumbers of specific resource types

Quantitative and qualitative characteristicsQuantitative and qualitative characteristics

2323


Dealing with Change and UncertaintyDealing with Change and Uncertainty

•• Vagueness in objectives and materialsVagueness in objectives and materials

•• Issues with performance source dataIssues with performance source data

•• Changing requirements and prioritiesChanging requirements and priorities

•• Technical and managerial changesTechnical and managerial changes

•• Appropriate modes of analysisAppropriate modes of analysis

2424


•• Vagueness in project objectives creates Vagueness in project objectives creates problems in defining scope and requirementsproblems in defining scope and requirements

Objectives of some projects and initiatives are poorly Objectives of some projects and initiatives are poorly defineddefined

•• Measurement is a matter of definitionMeasurement is a matter of definition

•• You can not measure what you can not defineYou can not measure what you can not define

•• It is critical to clarify objectives It is critical to clarify objectives

•• One approach: GOne approach: G--QQ--M methodologyM methodology

Establish database and collection requirements for Establish database and collection requirements for projectsprojects

•• Mechanisms are available, standards are notMechanisms are available, standards are not

•• Data is lifeblood of IT and must be captured and Data is lifeblood of IT and must be captured and effectively usedeffectively used

•• The problem of participation The problem of participation ––> exchange theory> exchange theory

2525


•• Issues with performance source data:Issues with performance source data:conflicts between benchmarks and realityconflicts between benchmarks and reality

Much available source material isMuch available source material is unreliableunreliable

•• No consistent methodologies or documentationNo consistent methodologies or documentation

•• Lack of a standard chart of accounts for software costsLack of a standard chart of accounts for software costs

•• Lack of engineering standards for software products, Lack of engineering standards for software products, processesprocesses

•• It is frequently necessary to fill in factual gaps with It is frequently necessary to fill in factual gaps with reasonable placeholder valuesreasonable placeholder values

•• Ultimately the most important values are localUltimately the most important values are local

Dealing with availability of performance source dataDealing with availability of performance source data•• Insufficient data Insufficient data –– attempt limited direct collectionattempt limited direct collection

•• Excessive volume Excessive volume -- sample source data for analysissample source data for analysis

•• Do not blindly accept nonDo not blindly accept non--equivalent benchmarksequivalent benchmarks

2626


•• Changing requirements and prioritiesChanging requirements and prioritiesRequirements changes impact projects and programsRequirements changes impact projects and programs

•• Changes are inevitable and consequentialChanges are inevitable and consequential

•• Rate of change and general volatility are critical issuesRate of change and general volatility are critical issues

•• Prepare to negotiate change impacts with customersPrepare to negotiate change impacts with customers

•• Manageable with effective measurementManageable with effective measurement

Changes in direction impact plans and programsChanges in direction impact plans and programs•• Unpredictable changes in priorities and budgetUnpredictable changes in priorities and budget

•• Result is wholesale changes to project & program plansResult is wholesale changes to project & program plans

•• Prepare to adapt with traceability and flexibilityPrepare to adapt with traceability and flexibility

•• Measurement should be as adaptive as the workMeasurement should be as adaptive as the work

•• Strategy should include design for reusability to Strategy should include design for reusability to preserve valuepreserve value

2727


•• Change in technical and managerial landscapeChange in technical and managerial landscape-- impacts to project measurement and outcomesimpacts to project measurement and outcomes

Technical changes Technical changes •• High risk in short termHigh risk in short term

•• Predictable and manageable in the long termPredictable and manageable in the long term

•• Tech changes sometimes affect measurement processesTech changes sometimes affect measurement processes

•• May require different measurements and experience setMay require different measurements and experience set

Managerial changesManagerial changes•• Managerial changes affect teamwork, objectives, visionManagerial changes affect teamwork, objectives, vision

•• Any staff change may be disruptive to workAny staff change may be disruptive to work

•• Usually disruptive to measurement effort Usually disruptive to measurement effort –– priorities shiftpriorities shift

•• Impacts may cause project /program cancellation or lossImpacts may cause project /program cancellation or loss

2828


•• Use of appropriate modes of analysis Use of appropriate modes of analysis ––understanding measurement requirementsunderstanding measurement requirements

Build the model to fit the purposeBuild the model to fit the purpose•• Software packages, development, maintenanceSoftware packages, development, maintenance

•• Process improvement, organizational change Process improvement, organizational change

•• Specific strategies for acquisition, other purposesSpecific strategies for acquisition, other purposes

•• Different perspectives apply to different measurement Different perspectives apply to different measurement objectivesobjectives

•• Design flexible measurement processes to adaptDesign flexible measurement processes to adapt

Model analysis to an appropriate level of detailModel analysis to an appropriate level of detail•• Proposal versus program versus projectProposal versus program versus project

•• Comparative versus budgetary needsComparative versus budgetary needs

•• Project versus processProject versus process

2929

Final ThoughtsFinal Thoughts

•• Defensibility relies on a logical approachDefensibility relies on a logical approachMust be believable, repeatable, backed by historyMust be believable, repeatable, backed by history

•• Assumptions: key to measurement approachAssumptions: key to measurement approachState & revise as assumptions change or resolveState & revise as assumptions change or resolve

•• The The ““funnel of uncertaintyfunnel of uncertainty””Wide at first, gradually narrowsWide at first, gradually narrows

•• No formulaic answer fits every problemNo formulaic answer fits every problemAdapt, extend, improvise, overcomeAdapt, extend, improvise, overcome

Ultimately, itUltimately, it’’s all about the requirement to define success s all about the requirement to define success and to know how close you come to achieving itand to know how close you come to achieving it

3030


Q & A

Contact Info:Terry Vogt

Booz Allen Hamilton8283 Greensboro Drive

McLean, VA 22102703-377-4567 [email protected]

Date post:	25-Mar-2022
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

Program Management Program Management with Requirements ...

Documents