1Arroyo CenterR

UXO Risk Assessment Methods: Critical Review

Jacqueline MacDonald, Debra Knopman, J. R. Lockwood, Gary Cecchine,

Henry Willis

RAND

2Arroyo CenterR

Briefing Outline

1. Need for UXO risk assessment methods Prioritization Site-specific assessment

2. RAND review of existing methods: tasks, approach

3. Design features and limitations of existing methods

4. RAND recommendations for improving UXO risk assessment

Prioritization Site-specific assessment

3Arroyo CenterR

Need for UXO Risk Assessment

Prioritization: mandated by Congress Site-specific assessment

Alternative response options have enormous cost implications

Total current cost estimate of $14 billion assumes mag-and-flag approach

Alternative approaches proposed by regulatory agencies could cost much more

DOD needs to understand risk reduction differences among alternative approaches

4Arroyo CenterR

Alternatives for UXO Response

Surface clearance only Scan with metal detector, and excavate each

anomaly to a specific depth (e.g. 2 ft, 3 ft) Scan, excavate each anomaly, scan bottom of

hole, and excavate again if anomaly is detected Scan and excavate anomalies, and then repeat the

process two or more times Excavate the entire site in one-foot lifts to depths

of 2 ft, 4 ft, or more; sift the excavated soil to remove UXO

5Arroyo CenterR

Alternatives Proposed at Ft. X

Scan land (surface & 1ft) Excavate entire site to 2 ft

Scan land (2ft level) Excavate entire site to 3 ft

Scan land (3ft level) Excavate entire site to 4 ft

Excavate anomalies found

Excavate anomalies found

Excavate anomalies found

Scan land (4ft level) Excavate anomalies found

Scan land with best available

technology (at surface, one scan

effort)

Provide construction support to property

developers

Excavate all anomalies found to maximum depth (4

feet)

Army’s Preferred Approach

State EPA’s Preferred Approach

6Arroyo CenterR

Cost Differences Among Ft. X Options Are Hundreds of Millions of Dollars

0

100

200

300

400

500

600

700

800

900

1000

0 1 2 3 4

Clearance Depth

Cos

t in

Mill

ions

Army State

7Arroyo CenterR

Briefing Outline

1. Need for UXO risk assessment methods Prioritization Site-specific assessment

2. RAND review of existing methods

3. Design features and limitations of existing methods

4. RAND recommendations for improving UXO risk assessment

Prioritization Site-specific assessment

8Arroyo CenterR

RAND Tasks

Client: Army Chief of Staff for Installation Management

Tasks:

1. Conduct a preliminary analysis of ongoing efforts in UXO risk assessment, including:• Ordnance and Explosives Cost-Effectiveness

Risk Tool• Interim Range Rule Risk Methodology• Ordnance and Explosives Risk Impact

Analysis• Risk Assessment Code

9Arroyo CenterR

Tasks, continued

2. Study methods used by the Department of Energy, National Aeronatuics and Space Administration, and others to evaluate and measure risk of low-probability and high-consequence events.

3. Recommend how the Army could develop a risk assessment/risk management protocol for UXO sites.

10Arroyo CenterR

Study Approach

1. Develop criteria for a technically sound risk assessment, based on risk assessment literature survey and consultations with experts

2. Read all documentation for available methods; test software if available

3. Interview method developers

4. Evaluate extent to which each method satisfies the evaluation criteria

11Arroyo CenterR

Evaluation Criteria Are in Three Categories

12Arroyo CenterR

Briefing Outline

1. Need for UXO risk assessment methods Prioritization Site-specific assessment

2. RAND review of existing methods

3. Design features and limitations of existing methods

4. RAND recommendations for improving UXO risk assessment

Prioritization Site-specific assessment

13Arroyo CenterR

IR3M Design

14Arroyo CenterR

Operation of the Explosives Safety Risk Tool

(Add all slides from UXO/Countermine Forum briefing)

15Arroyo CenterR

OECert Design

16Arroyo CenterR

RAND Evaluation of OECert

17Arroyo CenterR

OERIA Design

18Arroyo CenterR

RAND Evaluation of OERIA

19Arroyo CenterR

RAC Design

20Arroyo CenterR

RAND Evaluation of RAC

21Arroyo CenterR

Natural and Cultural Resources Bank Design

22Arroyo CenterR

RAND Evaluation of Natural and Cultural Resources Bank

23Arroyo CenterR

[Insert red light/green light slide]

24Arroyo CenterR

Briefing Outline

1. Need for UXO risk assessment methods Prioritization Site-specific assessment

2. RAND review of existing methods

3. Design features and limitations of existing methods

4. RAND recommendations for improving UXO risk assessment

Prioritization Site-specific assessment

25Arroyo CenterR

[Insert slides on prioritization system from final briefing]

26Arroyo CenterR

Briefing Outline

1. Need for UXO risk assessment methods Prioritization Site-specific assessment

2. RAND review of existing methods

3. Design features and limitations of existing methods

4. RAND recommendations for improving UXO risk assessment

Prioritization Site-specific assessment

27Arroyo CenterR

We Searched Other Agencies for Risk Assessment Models

[Use slide from countermine forum briefing, but modify “scenarios approach” to read “PRA”; also, place EPA first in list, and split FAA and NRC, and eliminate OSHA]

Then, insert slides 28, 29, 30 from countermine forum briefing

28Arroyo CenterR

Summary of Recommended Approach for Site-Specific Risk

Assessment

1. Use EPA Risk Assessment Guidace for Superfund methods to assess risks of munitions constituents

2. Develop probabilistic risk assessment method specific to UXO to assess explosion risks Development should be overseen by a

technical advisory committee The method should be independently peer

reviewed Template “trees” should be developed Trees then could be modified at individual

sites, with substantial stakeholder input

29Arroyo CenterR

Summary

Existing methods for UXO risk assessment do not satisfy criteria for technical credibility

New prioritization method should include a two-tier screen:

Tier 1: sort by explosion risks (using RRSE or HRS)

Tier 2: sort by constituent risks (using new method—possibly a modified RAC)

New site-specific assessment method should Use RAGS for constituent risks Use new PRA method for explosion risks

30Arroyo CenterR

[Insert quote used at UXO Forum]