Automating Test & Evaluation
of Portable Radiation Detection
Systems
Mr. Bryan Gerber
Pacific Northwest National Laboratory
Test & Evaluation Center
ITEA Annual Symposium, August 20th, 2015
2
Research Sponsor
Department of Homeland Security
Domestic Nuclear Detection Office (DNDO)
Systems Engineering & Evaluation Directorate
3
Background
Lead Engineer for Common Operation Response
Environment (CORE)TM Data Collection System
Radiation Detector Test & Evaluation for
Department of Homeland Security, Domestic
Nuclear Detection Office
Department of Defense, Defense Threat
Reduction Agency
Department of Energy, National Nuclear Security
Administration
4
Detectors of All Sizes
Straddle Carriers
Tractor Trailer Portals
Passenger Vehicle Portals
Human Portals
Backpacks
Handhelds
Pagers
Performance Testing
5
Characterize the performance of a detector
Verify compliance with international standards
Verify customer specific requirements
Perform repeated trials for statistical confidence
Provide data defensibility for procurement-sensitive
testing
Test: Dynamic Response
6
Dynamic Response Requirement
“The Backpack Radiation Detector shall alarm in 96
out of 100 trials… no later than 2 second after the
source passes the point of closest approach…”
- ANSI N42.53-2012 Section 6.3.2
Test: Slow Approach
7
Background Update Requirement
“…slowly approach at 0.5 m/s… The alarm shall
activate within 2s after the instrument reaches the test
position”
- ANSI N42.32-2006 Section 6.5
Test: Static Detection
8
Time to Alarm Requirement
“The instrument shall alarm in ≤2 s after exposure …
over a period of not more than 0.5 s”
- ANSI N42.32-2006 Section 6.2
Test: False Alarm
9
False Alarm Requirement
“The false alarm rate… shall be less than or equal to
one alarm activation in a 10 h period…”
- ANSI N42.48-2008 Section 6.2
The Old Way
10
Video review
Collection contains hundreds of hours of video
Video must be reviewed a second time to extract
data
Precision is limited by camera frame rate
Separate recording equipment required to extract
audible alarms
Manual timing
Accuracy does not meet requirements
Other timing metrics are difficult to verify
Automation Concept
Use inexpensive microcontrollers to automatically
monitor and record test data
Sensors monitor instrument alarm status
Sensors monitor lab equipment status
System automates collection of photo and video
data for defensibility purposes
11
Automation Benefits
Dramatically reduced labor hours
Increased data precision and accuracy
Reduced transcription errors by logging
events digitally
12
System Build
PCB Design Assembled PCB Integrated System
13
Physical Hardening
Field Ready
19 hour battery
Waterproof
Marine testing, October 2014
14
Test: Dynamic Response
15
Test: Static Detection
16
Sensor Types
Photoresistor: ON/OFF lights
Microphone: Audible alarms
Magnetic Sensor: Location of test
articles
Color Sensor: Red/green/blue light
to differentiate visible alarms
Touch Sensor: Trigger events on
button push
17
Sensor Locations
18
Sensor Performance
Sensor Type Response Time (ms)
Photoresistor 11
Microphone(Amplitude Only)
15
Microphone (Amplitude & Frequency)
100
Red/Green/Blue Color 11
Magnetic 11
Touch Sensor 11
19
Verify Response Time
Compare to frame by frame analysis
Record frame numbers for LED source, LED alarm
Compare to visual indications on detector
Examples: 23 frames per second (0.043s accuracy)
Sometimes we “beat” the visual indications
20
Comparing 2011 & 2014
Tested same class of instruments, Radiation Isotope
Identification Devices (RIIDs)
Received similar or identical models from same
vendors
Evaluated performance per ANSI N42 requirements
21
Repetitions Per Day
0
10
20
30
40
50
60
70
Completed Validated
Automated
Manual
22
Tradeoffs
Time added to dry runs & dress rehearsals
Additional setup time during testing
Judging time savings for automated vs. manual
collection for different types of tests
Test Type Video Capture Savings
Alarm TimingSavings
False Alarm Major Minor
Static Identification Major None
Dynamic Response Major Major
Detection Limits None None23
Example Analysis
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
SUT-A-1
SUT-A-2
SUT-A-3
SUT-A-4
SUT-A-5
SUT-B-1
SUT-B-2
SUT-B-3
SUT-B-4
SUT-B-5
SUT-C-1
SUT-C-2
SUT-C-3
SUT-C-4
SUT-C-5
Distance of Alarm (m)
Inst
rum
en
t
Slow Approach Alarm Distance
24
