A special colloquium on homeland security and ISyE

“Developing Usable Metrics for Venue Security”Dr. Paul Kantor

Honorary Associate, Industrial and Systems Engineering, University of Wisconsin-MadisonDistinguished Professor Emeritus, Rutgers University School of Communication and Information

Research Director at the Command, Control, and Interoperability Center for Advanced Data Analysis (CCICADA) at Rutgers

“Screening Commercial Aviation Passengers in the Aftermath of September 11, 2001”Dr. Laura Albert McLay

Associate Professor, Industrial and Systems Engineering, University of Wisconsin-Madison

Screening Commercial Aviation Passengers in the Aftermath of September 11, 2001

Laura Albert McLay, PhDAssociate Professor

University of Wisconsin-MadisonIndustrial and Systems Engineering

laura@engr.wisc.edu@lauramclay

http://punkrockOR.wordpress.com/

Collaborators: Sheldon Jacobson, Alex Nikolaev, Adrian Lee, John Kobza

Jacobson’s research on aviation security has been supportedin part by the US National Science Foundation (CMMI-0900226)

Executive summary

Better security is achieved by targeting scarce screening resources at the “riskiest” passengers and doing less screening on most passengers.

Underscreening / Overscreening occur given the uncertainty associated with risk assessment and limited security resources available.

Right Screening is ideal, but challenging to attain for all passengers.

TSA Precheck implicitly focuses on underscreening, which is why it makes the air system safer, in low risk, cost-constrained environments.

Passenger screening: Past, present, and future

One hijacking

encourages

copycats:

Hijackings are

contagious

Metal detectors and

X-ray machines go in

place

A brief history of passenger screening

• Dawn of time until 1970• Not much!

• 1970• Surveillance equipment, air marshals

• Feb 1972• Armed guards to make people feel safe• FAA adopted emergency rules to screen passengers

by at least one method including behavioral profile, metal detector, and ID check

• Dec. 1972• Metal detector / magnometer

• 1996• Checked baggage for high-risk passengers screened

for explosives (run by airlines) – CAPPS • Positive passenger baggage matching rejected

• Sept. 2001• Times are a changin’

Security is a moving targetChanges in passenger and baggage screening

• November 2001 – Aviation Transportation and Security Act• Created the TSA

• Required all checked baggage to be screened for explosives, Dec. 2002 deadline

• Driven by availably technology

• December 2001• Remove shoes

• 2002 +• CAPPS II, Secure Flight, etc. for risk-

based screening

Security is a moving targetChanges in passenger and baggage screening

• August 2006• No more liquids after liquid

explosives used in a bomb attempt

• February 2010• Random use of explosive trace

detection for carry on baggage in response to Christmas bombing attempt

• September 2012• Less screening for seniors (75+) and

children (<12)

• December 2013• TSA Precheck for reduced security

Why are homeland security problems good ISyE problems?

• Limited resources

• Passenger risk assessments

• Tradeoffs among criteria (efficiency, security, cost)• Note: TSA has a goal of <10 minutes waiting for screening

• System and goals are always changing

We will always have security challenges, and ISyE tools will always help us address some of these challenges.

Security often improved through deterrent effect, not detection.

Framework

• Passengers/bags screened by series of devices grouped in classes• System response a function of device responses

• Passengers check-in sequentially

• Passengers assigned to one of M classes upon check-in

Screening Assessments

Passenger risk assessments have been used since 1996.

What is known: As risk increases, likelihood of a security threat outcome increases.

Most passengers are low-risk.

Risk-based security: Captured in the Dynamic Aviation Risk Management System (DARMS) paradigm.

11

Retrospective Security & Resource AllocationKnow everyone’s risk before they enter security screening; allocate security resources to match risk.

Assumptions:

Security resources are limited.

Screening procedures make errors

* False alarms, False clears.

Security resources allocated to a passenger match the retrospective security resource allocation.

Multilevel Passenger Screening Problem (MPSP)* uses integer programming to maximize the security of the system, subject to

1) security resource constraints / limitations,

2) performance limitations of these resources,

3) security devices / procedures may be assigned to multiple security classes.

* McLay, L.A., Jacobson, S.H., Kobza, J.E., 2006

Screening Procedure Reality

Screening decisions must be made in real-time.

* Control Theory models1

* Markov Decision Processes2

Based on the Sequential Stochastic Assignment Problem (SSAP).

1 Lee, A.J., McLay, L.A., Jacobson, S.H., 2009

2 McLay, L.A., Jacobson, S.H., Nikolaev, A.G., 2009

Each passenger has a risk profile

* Used to determine the security resources allocated to their screening.

Three possible scenarios:

Right ScreeningUnder ScreeningOver Screening

Real-time simulation of optimal policy

Time

Passenger risk

Conditional distribution of passenger risk assessment values

Passenger risk

Over / Under ScreeningSecurity resources allocated to a passenger do not match the retrospective security resource allocation

Under too few resources are allocated to that passenger

Over too many resources are allocated to that passenger

Consequences: security resources not utilized correctly.Will the system be more vulnerable?

Over/under-estimating risk

Overestimating risk*: True risk level () < estimated risk level (’).

Understimating risk: True risk level () > estimated risk level (’).

* The tendency is to overestimate risk 16

Discussion

17

When risk is overestimated, high value security resources get used on low risk passengers, which may leave fewer high value security resources available for high risk passengers.

When risk is underestimated, high value security resources get used on high risk passengers, which targets more closely the high value security resources for high risk passengers

Resource matchingSensitivity analysis with respect to perceived risk in the system

Overestimate risk

True risk level < estimated risk level.

Underestimate risk

True risk level > estimated risk level.

Estimated risk level 𝜇′

Key ObservationsIn low risk environments, overestimating risk leads to a greater mismatch between security resources and passenger risk.

In low risk environments, underestimating risk leads to lower levels of underscreening and (for M small) lower levels of overscreening, compared to overestimating risk.

* Better resource matching

* TSA PreCheck

* Giving TSA Officers the (limited) flexibility to assign passengers to PreCheck lanes is an indirect way to mimic underestimating risk

19

Gaming Strategies

BlockingOvertaxing / decoysTimingTrial and Testing

All can disrupt the system in limited resource environments

20

Bad intentions

We are trying to prevent attacks

Is the goal to identify non-threat passengers with banned items or threat passengers with bad intentions (and no banned items)?

Risk based security focuses on the latter

21

Security systems

http://www.tsa.gov/about-tsa/layers-security

Thank you!

Laura Albert McLay, PhDAssociate ProfessorUniversity of Wisconsin-MadisonIndustrial and Systems Engineeringlaura@engr.wisc.edu@lauramclayhttp://punkrockOR.wordpress.com/

Blog posts:

• Aviation security, there and back again

• Aviation security: is more really more?

23

24

ReferencesJacobson, S.H., McLay, L.A., Kobza, J.E., Bowman, J.M., 2005, “Modeling and Analyzing Multiple Station Baggage Screening Security System Performance," Naval Research Logistics, 52(1), 30-45.Jacobson, S.H., McLay, L.A., Virta, J.L., Kobza, J.E., 2005, “Integer Program Models for the Deployment of Airport Baggage Screening Security Devices," Optimization and Engineering, 6(3), 339-359. McLay, L.A., Jacobson, S.H., Kobza, J.E., 2006, “A Multilevel Passenger Screening Problem for Aviation Security,” Naval Research Logistics, 53(3), 183-197.McLay, L.A., Jacobson, S.H., Kobza, J.E., 2007, “Integer Programming Models and Analysis for a Multilevel Passenger Screening Problem, “IIE Transactions, 39(1), 73-81.Nikolaev, A.G., Jacobson, S.H., McLay, L.A., 2007, “A Sequential Stochastic Security System Design Problem for Aviation Security,” Transportation Science, 41(2), 182-194.McLay, L.A., Jacobson, S.H., Kobza, J.E., 2008, “The Tradeoff between Technology and Prescreening Intelligence in Checked Baggage Screening for Aviation Security,” Journal of Transportation Security, 1(2), 107-126.McLay, L.A., Jacobson, S.H., Nikolaev, A.G., 2009, “A Sequential Stochastic Passenger Screening Problem for Aviation Security,” IIE Transactions, 41(6), 575-591 (2009 Outstanding IIE Publication Award).Lee, A.J., McLay, L.A., Jacobson, S.H., 2009, “Designing Aviation Security Passenger Screening Systems using Nonlinear Control,” SIAM Journal on Control and Optimization, 48(4), 2085-2105.McLay, L.A., Lee, A.J., Jacobson, S.H., 2010, “Risk-Based Policies for Aviation Security Checkpoint Screening,” Transportation Science, 44(3), 333-349.