Criticality Analysis & Risk Assessment: Determining High Risk Requirements CARA Process Methodology 1
Transcript
Slide 1
Slide 2
CARA Process Methodology 1
Slide 3
Introduction Objectives Testing Intake Process CARA Scoring
Risk Analysis and Testing Scope Report Challenges 2
Slide 4
3 Paul Shovlin Checkpoint Technologies Director of Professional
Services IV&V Functional Test Manager, Department of Veterans
Affairs
Slide 5
4
Slide 6
Understand the Testing Intake Assessment (TIA) process. Provide
knowledge and understanding of Criticality Analysis and Risk
Assessment (CARA) principles and practices. Understand the purpose
and content of the Risk Analysis and Testing Scope Report (RATSR).
Share our challenges. 5
Slide 7
September 2010 mandate that all application release cycles
complete from development to Initial Operating Capability (IOC)
within 6 months, not leaving much time if any for IV&V testing.
150+ Major application project releases each calendar year. New
approach to IV&V needed. CARA was being used successfully at
NASA. 6
Slide 8
7
Slide 9
8 Testing Intake Assessment Form CONOPS Requirements System
Design CONOPS Requirements System Design Submit Testing Intake
Assessment Read, Review Docs & Formulate Questions SMEs Answer
Questions Prepare CARA Worksheet Perform CARA Create & Review
RATSR Determine Services Testing Workload Form Update Project
Schedule Update Project Schedule Update Testing Workload Forecast
Project Management Plan Project Schedule Project Management Plan
Project Schedule Assign CARA Resources CARA Worksheet Updated CARA
Worksheet Updated CARA Worksheet RATSR
Slide 10
Risk is the likelihood of failure. Criticality is the measure
of potential impact of failure. Risk-Based Testing is a type of
software testing that prioritizes the tests of features and
functions based on their criticality and likelihood of failure.
Risk-based testing determines which test activities will be
completed for the iteration. It is a transparent testing
methodology that tells the customer exactly which test activities
are executed for every feature. 9
Slide 11
Criticality Analysis and Risk Assessment Standardized risk
assessment methodology. Features are consistent with PMIs Risk
Management assessment philosophy, PMBOK Guide. The uniqueness is in
the process of determining the impact and probability. Standardized
criteria for determining risk values. CARA does not tell us how to
test. 10 IMPACT Probability LowModerateHigh Catastrophic Critical
Moderate Low
Slide 12
11
Slide 13
Criticality is broken down into 3 categories: Performance &
Operations Safety Cost of Failure/Impact to Schedule Requirements
for each category are scored on a scale of 1-4 1 = Lowest 2 =
Moderate 3 = Critical 4 = Catastrophic 12
Slide 14
Risk is broken down into 5 categories: Complexity Technology
Maturity Requirements Definition and Stability Testability System
Characterization Requirements for each category are scored on a
scale of 1-3 1 = Low 2 = Moderate 3 = High 13
Slide 15
Scores for Risk and Criticality Categories are separately
averaged to determine a weighted value The weighted value scores
are then multiplied to determine the IV&V Analysis Level (IAL)
Threshold IALs are broken down into 4 categories: Minimal:
1CARA