Schedule Risk Assessment Overview

Schedule Risk Assessment (SRA) OverviewJuly 2013

NAVY CEVM

#Welcome to the Navy CEVM Training on Schedule Risk Assessments (SRA)

OutlineWhy is an SRA required?How does it work?Who performs the SRA?Setting the Target and ProbabilityReviewing the Results and Identifying TradeoffsDocumenting the Results

#Well begin this training module by discussing why an SRA is required and how it works. Well then discuss the roles and responsibilities for conducting an SRA, and the selection of the SRA target and probability goals. Well further discuss the review of SRA results and the identification and processing of tradeoffs. Well conclude with a discussion of SRA documentation. 2

Why the SRA Requirement?With hundreds if not thousands of similar relationships in a schedule, SRA helps evaluate the probability of completing the effort by a given deadlineTask 150% ProbabilityTask 250% ProbabilityTask 325% Probability

#An SRA is an effective PM tool, based on simple probability theory, that can be used to assess the probability of schedule completion. It is required in conjunction with the establishment of a new baseline, to include the start of the program, rolling wave planning, OTBs, or Single Point Adjustments. Additional SRAs may be required by the CDRL.

On this chart were zoomed in on three specific tasks. Tasks 1 and 2 are required to finish before task 3 can begin. Assume a 50% probability of completion for each estimated finish. Without any other factors, on time completion of task 3 has only 25% probability.

Task 1 Task 2 Task 3OntimeLateLateLate On-timeLateLateLateLateOntimeOntimeOntimeOr 25% probable.

Each task in the schedule has its own range of expected outcomes and probabilities. Compounded for all of the tasks in the schedule, delays can be significant. An SRA considers all of the relationships in the schedule and assesses the probability of achieving a given target. The output is a range of possible completion dates and associated probabilities for the milestone. 3How Does it Work?Monte Carlo Simulation is run to determine the probability of completion for a desired SRA target milestone(s) by a desired date(s):Completed for a specified number of iterations Increased number of iterations increases prediction accuracyBased upon a random sampling of outcomes bounded by user provided best case, worst case, and most likely outcomes for each task in the IMSIndividual estimate ranges are required for each critical, near critical, and high risk task. These estimates should be developed by the responsible Control Account Manager (CAM) or technical leadRemainder of tasks may be assessed in groups or globallyResults of the simulation may either be used to prompt changes in the schedule or to revise expectations and/or contract terms.

#Monte Carlo Simulation predicts the completion date of a target milestone by randomly assigning a duration for each task in the schedule from within that tasks predicted range of best, worst, and most likely outcomes. It can iteratively perform this action thousands of times in quick succession and tally the frequency with which each potential completion date is encountered, thereby producing a likelihood of completion for each date. The more iterations that are run, the more statistically significant the result.

Accurate Best, Worst, and Most Likely estimates are obviously vital to the accuracy and usefulness of SRA. There are various techniques for developing task duration estimates, but they must be developed with participation from CAMs and technical leads. Individual estimates must be considered and carefully crafted for the riskiest tasks. Less risky tasking may be more globally assessed. For example, consider a 20 day task that is among a group of less risky tasks that are assessed with a 10% band. It would have a best case duration of 18 days, expected duration of 20 days, and worst case duration of 22 days.

The results of the simulation should be used to make changes to the schedule that increase the probability of on time completion. Generally Monte Carlo Analysis is run multiple times before a successful outcome is achieved.4Who performs the SRA?The government sets the target milestone for the assessment (typically a program event)The contractor PM sets the desired probability (often with input from the government) and oversees the effortContractor CAMs and technical leads are responsible for developing the best case, worst case, and most likely duration estimatesContractor scheduler is responsible for coordinating the effort and performing the Monte Carlo simulation.All parties participate in:Reviewing the results and correcting errorsIdentifying tradeoffsDocumenting the results

#Generally speaking, an SRA is conducted primarily by the contractor. The government is responsible for setting the target milestone(s) for the analysis, but the SRA effort is overseen by the contractor PM who generally sets the target probability. The bulk of work associated with an SRA is performed by the program scheduler, Control Account Managers, and Technical leads. The CAMs and technical leads are responsible for developing the best case, worst case, and most likely duration estimates, and the scheduler is responsible for collecting and processing this data as well as running the Monte Carlo simulations.All parties are responsible for reviewing the results of the simulation, correcting errors, identifying tradeoffs that lead to improved simulation results, and documenting the final results.5Setting the Target for the SRAThe government is responsibile for setting the SRA target date. The target date selected depends on the objectiveTypically it is recommended that both the end milestone and the current baseline be selected as SRA targetsThis allows for overall optimizationExamples IBRs End of contract and first milestoneRolling Wave End of contract and detailed period end milestoneOTB End of contract

#The default milestone for SRA analysis is of course contract completion; however, SRA may be run to any task, and the target should be selected based upon the objective. For example, the SRA may focus on the path to an intermediate program milestone or the end of a rolling wave planning window.6Setting the ProbabilityWhat does the Government deem an acceptable level of risk that delivery may be late? Is the project technical, schedule, or cost driven? Typically strict schedule or cost objectives drive the probability higher.General Guidance> 90% is not cost effective< 50% is very riskyFor most contracts, a probability of 50-80% strikes a reasonable balance between risk vs cost

#Risk tolerance and therefore the target probability for each contract and situation may be different. A low target probability is obviously not desirable for the amount of risk it assumes, but a high probability is often not affordable. The recommended default is 75%. This assumes an amount of risk that can be managed at a reasonable cost.

By establishing an IMS with a completion date that beats the contractual milestone by 10%, one can increase the chances of reaching a starting probability between 50-80%. When the SRA is complete this gap can be considered for schedule margin. Schedule margin should not be included in the schedule prior to the Monte Carlo Simulation as this can distort the results.7Review the Results - Histogram

This histogram plots the number of iterations which predict completion by any given date, as well as the associated probability of completion by that date

#8The exact output format will vary by tool, but a Risk Histogram is one of the two main products of an SRA. This particular output is from Risk+, a common tool that runs with MSP. The output is run for the milestone target and gives cumulative probabilities for a variety of dates. If the target milestone has an acceptable probability by the target date, then the schedule may be fine as is. Unfortunately, this is not usually the case, at least not on a first run. In the first pass of an ACAT 1 type program with a year or more, it is not uncommon to have the first probabilities under 1%.

In the example shown here, assume that the target date was 2/3. According to the table, completion by that date was only 5% probable. If the desired probability was 75%, then the expected delivery was 2/20.

Review the Results Tornado ChartsTornado Charts identify (by decreasing significance), the tasks whose duration estimates and logic are most likely to impact the target milestone completion dateThe two examples above demonstrate different but related metrics plotted for this purpose

#9The other main product of an SRA is a tornado chart, so named for the shape taken by plotting data points by decreasing significance from top to bottom. The benefit of a Tornado chart is the identification of tasks whose duration estimates and logic are most likely to impact the target milestone end date. Tornado charts can be run using a few different but related metrics. Essentially, they provide the PM with insight to which tasks and sections of the schedule to focus on in order to minimize risk and the overall project duration.The examples shown here were produced using Primaveras Schedule Risk Assessment Tool.

Identifying TradeoffsSRA analysis is an iterative process. After each simulation, the program team must identify the problem or risk areas, evaluate cost/schedule/technical tradeoffs of alternate solutions, implement the fix, and re-run the simulation until it can produce a schedule that achieves an acceptable level of probability by an acceptable completion date. Options include:Addition of resources to enable earlier task completionTask logic and sequencing may be reconsideredRisk mitigation activities may be funded in order to reduce task durationsRevised expectations or contractual milestone dates

#10As already mentioned, SRA is an iterative process. Monte Carlo analysis is often run many times before a valid result is reached. Between each run, problem areas are identified, tradeoffs between cost/schedule/and technical aspects are considered, and adjustments are made. As the PM and his team attempt to increase the chances of on-time completion, there are a standard approaches. First, resources may be added to enable earlier task completion. Second, task logic and sequencing may be reconsidered to allow more concurrency. Third, Risk mitigation activities may be funded in order to reduce task duration ranges. Finally, if a solution can not be reached that increases the probability, the target date may be reconsidered. Either way, a desired probability by a target date should be achieved before the SRA can be considered complete.Documenting ResultsDocument final probability of the target completion dateEnsure the risk list is updated for any newly identified risk items and associated mitigations and ensure that the mitigation activities are properly labeled and tracked in the IMSDocument any additional requested schedule refinement or adjustments to be addressed post-SRAIf the SRA is conducted during initial baselining, rebaselining, or rolling wave activity, take the opportunity to baseline the results if possible

#11Upon completion of the SRA, the final projections should be documented including the probability histogram and the tornado chart(s). The IMS and risk register should be updated for any risks and risk mitigation activities identified. Any post-SRA actions should also be documented.

Point of ContactNavy Center for Earned Value Management

(703) 695-0510

http://acquisition.navy.mil/acquisition_one_source/cevm

#