Potomac Management Group, Inc.1 Objective POD Estimation The Development of a Standard Method For...

Potomac Management Group, Inc. 1

Objective POD EstimationThe Development of a Standard MethodFor Gathering and Using Detection Data

R. Quincy Robe & Jack Frost


Presentation OutlineDefine and Describe a “detectability index”Show how it is used with other data to estimate PODDescribe a Procedure for doing detection experiments to determine a “detectability index”


The Detection ProcessA series of “glimpses” as the searcher moves through the environment containing the object.Detection with any one “glimpse” depends on the

Search Object (size, color, contrast, etc.)Environment (weather, terrain, vegetation, etc.)Search Resource (sensor and platform)Distance from the Resource to the Object


What is Probability of Detection (POD)?

Applies to some amount of area (e.g., a segment)Probability of detecting an object if presentPOD is a function of:

Effort (Resources, Search Speed, Time)Size of the Area coveredSearch object “detectability”


What is Effort?Total Distance traveled by searchers while searching in the segment

Effort = searcher speed x time x number of searchers

What is Area covered?Size of the area over which the searching effort is approximately uniformly spread


What is Detectability?

How can one measure or quantify how easy or hard it will be to detect a particular object with a particular type of resource (sensor) in a particular environment?


What about Maximum Detection Range?Easy to measure directly.Measures how far from the sensor an object can be detected by an alerted searcher who knows where to look.Does not address whether the object will be detected within that range. Does not measure how much detecting can be expected from a searcher (sensor). No simple, predictable correlation with detection performance.


What about direct estimation?Humans are very poor at estimating probabilities of any kind.Compare:

How many of 10 objects would you have found?How many of 10 objects could you have missed?

No such thing as “one size fits all” POD for everything from small clues to large objects.Direct estimation = Wild Guess


Effective Sweep Width (Koopman)Cannot be measured directlyIs an objective measure of detectibility

Large value => easy to detectSmall value => hard to detect

Depends on the characteristics ofSearcher/Sensor (What we are searching with.)Search Object (What we are searching for.)Environment (What we are searching in.)

Terrain, Vegetation, Weather, etc.

Has units of length (feet, meters, miles, etc.)


A Uniform Random Distribution


Effective Sweep Width

Number detected = 40.Number missed within sweep width = 0.

Number detected outside sweep width = 0.


(Unrealistic Ideal Sensor Making a Clean Sweep)



Number detected = 40.Number missed within sweep width = 16.

Number detected outside sweep width = 16.


Max Detection Range

(More Typical Sensor)


Effective Sweep Width NotesIn both of the previous examples, there were

The same object density (# of objects/unit of area),The same length of searcher track, andThe same number of objects detected (40).

Therefore,The effective sweep widths are also the same.

Effective sweep width represents the expected amount of detection.


Lateral Range (Koopman)Distance to right or left of sensor at the closest point of approach (CPA)Lateral range curve



Key to Improved Search Planning and EvaluationImproves POD Estimation

Allows us to Objectively Relate POD to Effort ExpenditureHas both Predictive and Retrospective ValueMore Accurate and Reliable than Subjective EstimatesBased on Observable Factors

Improves Effort AllocationMakes known, proven (mathematical) techniques availableImproves conceptualization of the search problem

Southern CaliforniaSouthern California

Southern CaliforniaSouthern California

Western Washington StateWestern Washington State

Western Washington StateWestern Washington State


Objective POD EstimationFor a searched segment

Effort = z = Total Distance Searchers Cover = search speed time number of searchersEffective Sweep Width = W from detection experimentsArea Effectively Swept = z W

Coverage = C =

POD = 1 – e-C (Koopman)

Area Effectively Swept

Area of Searched Segment


POD vs. Coverage Graph (Koopman)POD vs. Coverage

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0

Coverage

PO

D

POD versus Coverage


“Uncorrected” Effective Sweep WidthsIn Nautical Miles For Aerial Search Over Land (IAMSAR Manual)

Meteorological Visibility (Nautical Miles)

Search ObjectAltitude

(Feet AGL)

3 5 10 15 20

Person 500 0.4 0.4 0.5 0.5 0.5

1000 0.4 0.4 0.5 0.5 0.5

Vehicle 500 0.9 1.3 1.3 1.3 1.3

1000 1.0 1.4 1.4 1.5 1.5

Small Aircraft 500 1.0 1.4 1.4 1.4 1.4

1000 1.0 1.5 1.5 1.6 1.6

Large Aircraft 500 1.2 2.0 2.2 2.2 2.2

1000 1.8 2.7 3.0 3.0 3.0


Effective Sweep Width Correction FactorsFor Aerial Search Over Land (IAMSAR Manual)

(Multipliers)

Search Object15-60% vegetation

or hilly60-85 % vegetation

or mountainous Over 85% vegetation

Person 0.5 0.3 0.1

Vehicle 0.7 0.4 0.1

Small Aircraft 0.7 0.4 0.1

Large Aircraft 0.8 0.4 0.1


Sweep Width Issues for Ground Search

Too many different types and combinations of terrain, vegetation, search objects for a “universal” set of sweep width tables.Each locale needs sweep widths only for its area of responsibility, typical search objects, etc.Solution: Develop a standard, practical, and scientifically based procedure for local resources to use when developing sweep width estimates.


The Logan, West Virginia

Demonstration Project


Project Support

Sponsored by the U. S. National Search and Rescue Committee (NSARC)Funded by Department of Defense (NSARC member)Contract administered by U. S. Coast Guard (NSARC Chair) via the USCG Research and Development Center; performed by Potomac Management GroupEndorsed by NASAR and U. S. Air Force RCCHosted by Logan Emergency Ambulance Service Authority


Demonstration ProjectPrincipal Investigator: R. Quincy RobeLocation: Chief Logan State Park, Logan, WV Host: Roger Bryant, Director, Logan Emergency Ambulance Service Authority (LEASA)Participants: Attendees at Logan SAR Weekend on 15-16 June 2002Outstanding support and hospitality!


Demonstration Project ObjectivesDesign Practical Detection Experiment Procedures to determine Effective Sweep Width values for ground wilderness/rural searches.Supervise a Demonstration of the Procedures Using Ground SAR Personnel.Describe Method for Objectively Estimating POD from Effective Sweep Width, Effort, and Area.Report Results and Describe Future Work required to generalize their application.


Concept of Operations (Preparation)Select a typical area and typical search object types (no more than 3 types)Select track(s) for searchers to follow (for at least 1 hour—longer is better)Choose date, select participants, make logistic arrangements, set up scheduleObtain/construct search objects (≥ 10 of each)


Concept of Operations (Execution)Place objects at random locations along the track and random distances on either sideSend searcher/data recorder pairs along the track at timed intervals (to ensure separation)Searchers move at normal search speed and report all sightings of search objectsData recorders record searcher sighting reports and other pertinent dataCollect and analyze the recorded data


Chief Logan State Park

Experiment Area


Select Search Track

End I

End II

a 240 m

220 m

c 180 m

220 m

e230 m

f

280 m

g340 m

h290 m

80 m

b

d

Waypoints a to h were marked with flags.Approximate distances between waypoints are in meters.


Search Objects

Orange Glove

Garbage Bag


Determining Object LocationsUseful range of distances off track

Too close => Insufficient data for longer rangesToo far => Wasted detection opportunities

Useful range of distances along trackToo close => Frequent reinforcement => alertnessToo far => Track too long for reasonable time

Use Average Maximum Detection Range


Average Maximum Detection Range


Select Object PlacementRandomize

Distances along the trackDistances off trackRight or Left of trackObject types

Determine locations based on largest AMDRAverage separation along track of 3 AMDROff track up to 1.5 AMDR


Example of Object Locations(AMDR = 100 m)

Search Object Locations

Track IntervalAlong Track

LocationCross Track

LocationSearch Object

Type

Location #1 100 to 300 241 m 122 m right B

Location #2 400 to 600 442 m 47 m left A

Location #3 700 to 900 886 m 69 m right A

Location #4 1000 to 1200 1033 m 22 m left B

Location #5 1300 to 1500 1420 m 45 m left A

More locations To end of track Next location Next locationNext search object type


Search Object Location Zones

2 x AMDR

3 x AMDR

1.5 x AMDR

1.5 x AMDR

LateralRange


What is a Detection Opportunity?For the purposes of a detection experiment, a detection opportunity is defined as one complete pass by the search object. If there are 15 identical search objects of a given type and 30 searchers in an experiment, then there are a total of 15 x 30 = 450 detection opportunities for that type.Each detection opportunity has one of two results: Detection or Non-detection.


Important NotesWhen performing a detection experiment, it is important to understand that:The relationship between the searcher (sensor) and the search object during the window of detection opportunity must be captured, andKnowing when non-detection occurs is just as important as knowing when detection occurs.


Important NotesThe experiment is NOT a competitive eventThe experiment does NOT measure individual searcher proficiencyDo NOT tell searchers how many objects are present, how far off track, or give any other hintsDO Collect additional data (e.g., weather, time of day, terrain and vegetation descriptions, searcher training/experience data, etc.) for later analysis


Perform ExperimentSecretly Place Objects at Selected LocationsSend Searcher/Data Recorder Pairs along the Selected Track at Timed IntervalsCollect Completed Detection Data FormsRemove Objects at Experiment’s Conclusion(Discard data for objects not found.)Compile, Sort and Analyze the Detection Data


Detection Log

Detection Log

TIME

EVENT

(Start, waypoint, detection, etc.)

Clock Bearing

(12 o’clock-Ahead as if track was

straight)

Est. Distance

(Indicate units used)

Comments?

0830 Start 0853 Black Bag 10 o’clock 50 m In brush

Date and Time Direction of Travel Location (clockwise/counter clockwise)

Name - Recorder Page Name - Searcher of


Calculate Sweep WidthUse the following property of sweep width:

The number of detections outside a swath one sweep width wide centered on the searcher’s track equals the number of missed detections inside that swath.Equivalently, the number of detections at lateral ranges greater than one-half the sweep width value are equal to the number of missed detections at lateral ranges less than one-half the sweep width value.


Logan Demonstration Statistics32 Searchers Participated12 Orange Gloves were placedGlove AMDR = 19 meters32 x 12 = 384 Detection Opportunities9 Black Garbage Bags were placedBag AMDR = 25 meters (1.5 x 25 = 37.5 meters)32 x 9 = 288 Detection Opportunities


Consolidated Detection Data

Orange Glove Lateral

Ranges

Orange Glove

Detections

Orange Glove Misses

Cumulative Non-

Detections From Zero

Cumulative Detections

From Infinity

Lateral Range Curve (POD)

Area Under

Half LRC

Area Under LRC (W)

Estimate by Cumulative Counts of "Hits" and "Misses"

(W)

Total Detection

Opportunities at Lateral

Range

Average Maximum Detection

Range (Rain

Dance)

2 23 9 9 179 71.88% 1.44 32 14

5 25 7 16 156 78.13% 3.69 32 30

9 32 0 16 131 100.00% 7.25 32 28

14 34 30 46 99 53.13% 11.08 64 24

17 29 3 49 65 90.63% 13.23 32 11

21 6 26 75 36 18.75% 15.42 36.33 32 12

24 3 29 104 30 9.38% 15.84 32 12

31 0 32 136 27 0.00% 16.17 32 18

32 2 62 198 27 3.13% 16.19 64 18.63

41 25 7 205 25 78.13% 19.84 39.69 32 1.95Check Sums 179 205 36.33 384

18.1636364


Orange Glove Sweep Width

Orange Glove Detection Data—W = 36 meters.(Crossing point equals one-half effective sweep width value.)

Orange Glove Half Sweep Width Estimator

136

25

179

156

131

99

65

3630 27 27

205

916 16

46

49

75

104

198

0

50

100

150

200

250

0 5 10 15 20 25 30 35 40 45

Lateral Range in Meters

Cu

mu

lati

ve D

ete

cti

on

s a

nd

No

n-D

ete

cti

on

s

Cumulative Detections from infinity

Cumulative Non-Detections from zero

2 9 14 17 21 24 31 32 41

1/2 W = 18.16 m

(AMDR = 25 m)(12 Gloves, 32 Searchers)


Orange Glove Half Lateral Range Curve

Orange Glove Half Lateral Range Curve—W = 40 meters(Areas under this portion equal one-half effective sweep width value.)

Orange Glove Half Lateral Range Curve

78.13%

90.63%

100.00%

18.75%

9.38%

3.13%

0.00%

53.13%

78.13%

71.88%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20 25 30 35 40 45


PO

D

Shaded Areas = 19.84 meters


Orange Glove Modified Sweep Width

Modified Orange Glove Detection Data—W = 33 meters.(Crossing point equals one-half effective sweep width value.)

Orange Glove Half Sweep Width Estimator(Modified data for lateral range of 41 meters.)

136

0

154

131

106

74

4011

5 2 2

230

916 16

4649

75

104

198

0

50

100

150

200

250

0 5 10 15 20 25 30 35 40 45


Cu

mu

lati

ve D

ete

cti

on

s a

nd

No

n-D

ete

cti

on

s Cumulative Detections from infinity

Cumulative Non-Detections from zero

2 9 14 17 21 24 31 32 41

1/2 W = 16.27 m


Orange Glove Modified Half LRC

Modified Orange Glove Half Lateral Range Curve—W = 33 meters.(Areas under this portion equal one-half effective sweep width value.)

Orange Glove Half Lateral Range Curve(Modified data for lateral range of 41 meters.)

90.63%

0.00%0.00% 3.13%

9.38%

18.75%

53.13%

100.00%

78.13%

71.88%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20 25 30 35 40 45


PO

D

Shaded Areas = 16.33 meters


Black Bag Sweep WidthBlack Bag Half Sweep Width Estimator

162

33

43

72

98

130

193

225

63

32107

1110

0

50

100

150

200

250

0 5 10 15 20 25 30 35 40


Cu

mu

lati

ve D

ete

cti

on

s a

nd

No

n-D

ete

cti

on

s

Cumulative detections from infinity

Cumulative non-detections from zero

1 3 6 12 21 26 37

1/2 W = 28.95 m

Black Garbage Bag Detection Data—W = 58 meters.(Crossing point is equal to half sweep width value.)

(AMDR = 25 m)(9 Bags, 32 Searchers)


Black Bag Lateral Range Curve

Black Garbage Bag Half Lateral Range Curve—W = 53 meters.(Area under this portion equals one-half effective sweep width value.)

Black Bag Half Lateral Range Curve

90.63%

100.00%

96.88%

100.00%

100.00%

81.25%

31.25%

51.56%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20 25 30 35 40


PO

D

Shaded Area = 26.71 meters


Lessons LearnedAMDR did not work well

Poor choice of location?Poor technique by investigators?Should have been repeated several times in different locationsMay need to use maximum, rather than average maximum detection range

Need steady flow of searcher/data recorder pairs


Future WorkValidate and refine detection experiment procedures in 3 different venues with different SAR groups and personnel during the next year.Publish the refined procedures and make them available upon request.Extend techniques to include aerial search over land (CAP, CASARA, etc).Develop more advanced search planning methods appropriate for the land SAR community.


Future Work (continued)

Develop functional requirements for software tools to support land SAR search planning.Survey existing software packages for synergistic opportunities.Develop software (modules) to support land SAR search planning functions.


ConclusionsA practical detection experiment procedure is feasible.Effective sweep width results make scientifically proven search planning methods available for use in land SAR.Objective, accurate, reliable POD estimation is possibleMore nearly optimal resource allocation can be done

Increase probability of success (POS) at maximum rate.Minimize mean time to find survivors.Save more lives.Minimize risks to searchers through reduced exposure times.Minimize costs through shorter searches on average.


Conclusions (continued)Effort needed is comparable to a SAREX.No special skills, tools or equipment required (although some items would be helpful).Data should be archived at a central site.Additional data gathered will support later analyses for important secondary effects For example, correction factors to extend usability of effective sweep width data to situations other than those of the experiments.


Potomac Management Group, Inc.510 King Street, Suite 200Alexandria, VA 22314Attn: J. R. Frost703-836-1037 or 202-267-6702 (USCG)E-mail: [email protected] or

[email protected]

THANK YOU!

Date post:	18-Dec-2015
Category:	Documents
Upload:	barry-elliott
View:	214 times
Download:	1 times

Potomac Management Group, Inc.1 Objective POD Estimation The Development of a Standard Method For...

Documents