Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

The challenges of using an intrusion detection

system: is it worth the effort?

Rodrigo Werlinger, Kirstie Hawkey, Kasia Muldner, Pooya Jaferian, Konstantin Beznosov

University of British Columbia, Canada

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)2

Werlinger, R., Muldner, K., Jaferian, P., Hawkey, K., Beznosov, K.

MotivationLiterature

“This task was based upon the monitoring and analysis phase of ID, the most time-consuming and cognitively challenging subtask in ID [9, 10, 23]”.

“Command Line or Pretty Lines? Comparing Textual and Visual Interfaces for Intrusion Detection” Thompson et al., CHI 2007

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)3

MotivationLiterature:Pre-processing phase of ID

is relatively easy“This task was based upon

the monitoring and analysis phase of ID, the

most time-consuming and cognitively

challenging subtask in ID [9, 10, 23]”.

Command Line or Pretty Lines? Comparing Textual and Visual Interfaces for Intrusion Detection” Thompson et al., CHI 2007

Our Perception:IDS configuration is *hard* Rodrigo’s current

experience deploying an IDS

His prior experiences in a telecommunications co.

Collective recollections of 1+ interview participants describing IDS configuration as a major hurdle

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)4

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)5

Intrusion Detection Systems (IDSs)

Intrusion detection phases: deployment, monitoring, analysis, response

Still need human intervention

ID requires high level of security expertise, organizational knowledge & collaboration

Most current research focuses on supporting monitoring + analysis phases (e.g., visualization, better detection algs)

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)6

Research questions

What do security practitioners expect from an IDS?

What are the difficulties they face when installing and configuring an IDS?

How can the usability of an IDS be improved?

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)7

Semi-structured interviews • 9/34 discussed IDS • 6 Academic, 1 Financial Services, 1

Scientific Services, 1 Consultant• 1 Security Manager, 1 IT Manager, 5

security, 2 general IT w/ security duties

Participatory observation • ~15 hours on IDS (~90 total)• Working with 2 senior Academic SPs

Approach

Results from Interviews

[an IDS is] “one of the most controversial [tools] – some really

love it, but some really hate it”

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)8

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)9

IDS Expectations: Advantages

Problem identification

Activities inside/outside firewall

Reduction of uncertainty

Could provide assurance of effectiveness of security measures

Monitoring with privacy

Decreased time pressure for maintenance

If using an Intrusion Prevention system

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)10

IDS Expectations: Disadvantages

Financial expense

Work and time required

• Tuning the system

Unreliability

• Buggy, dropped packets

Lack of clear utility

• Hard to see an improvement, often sit idle

Results from Participatory Observation History

• IDS installed 2 years prior in one network domain

• Crashed, memory space issues• Unclear whether problem was with setup or

newly added wireless

• No time to confirm exact cause• Decided to re-install from scratch on a

different network• This delayed for several months

• High workload, competing priorities

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)11

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)12

Issues deploying an IDS (1/5)

Deciding on the purpose of the IDSDeciding on the purpose of the IDS

1. Improve efficiency of monitoring

2. But also:• Statistics on network security

• Support for increasing security budget

Ultimately, (2) proved too complicated…

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)13

Issues deploying an IDS (2/5)

Integrating the IDS in the network

To connect the IDS, 2 ports were needed Wanted to use port-mirroring feature to

select traffic wanted to monitor

These requirements could not be realized

IDS installed in a less critical network

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)14

Issues deploying an IDS (3/5)

Configuration via IDS GUI

Quick tune option But inadequate for complex task:

• Can’t specify hard disk partitions

• No support for configuring IDS security settings (server firewall rules)

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)15

Issues deploying an IDS (4/5)

Distributed Environment

Extra overhead • Involvement of various organizational members

without security as a priority

Multiple stakeholders need to configure IDS• But IDS did not support fine-grained access

control

Compromise: less critical network, but autonomy

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)16

Issues deploying an IDS (5/5)

Usability / Utility Tradeoffs

Ideally IDS would have been deployed in critical network (utility high, usability low)

Hard to assess IDS utility without full deployment

• Unclear if large network domain more demanding

False positives vs. false negatives tradeoff

• Can’t tune until running

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)17

Considerations Before Deploying

• Show economic benefit to get buy-in• Minimize overhead costs (stakeholders)• Broad knowledge of organization & systems

Configuration &Validation

• Distributed environment • Initial configuration hurdle • Determine appropriate test bed

Ongoing Use

• Collaboration features• “A bit of smarts”• Reports for different stakeholders

Challenges throughout IDS deployment

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)18

Planning

IDSs not yet de facto tools

IDS deployment impacts many stakeholders

IDS utility must be clear, butuntil deployed and configured…..

Formalize via dedicated projectInvolve stakeholders

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)19

Configuration and validation

Configuration hurdle (rule customization)

Distributed environment

How to test the IDS (“all or nothing” tool)

Quick tuning

Flexible reporting

Support for finding test-bed

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)20

On-going usage

Detection of trends• “A bit of smarts”

IDS usage via various stakeholders

Artificial intelligence

Collaboration features

Flexible reports

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)21

Summary

Many factors will determine whether deploying an IDS is worth the effort

Challenges are present in all stages and not limited to technology

Tool support needed to help meet the challenges

More study needed to determine generalizability of our participants’ experiences

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)22

Thank you

hawkey@ece.ubc.ca

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)23

Challenges and recommendations

Why an IDS?

Perceptions of IDSs

Planning and installation

Broad and deep knowledge

Intensive collaboration

Representative Testebed

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)24

Technical and organizational challenges

Broad and deep knowledge

Intensive collaboration Representative Testbed Meaningful reports

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)25

Original slides that came right after the results

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)26

Planning

• Show economic benefit• Minimize costs• Detection efficient

Configuration &Validation

• Distributed environment • Initial configuration hurdle • Determine appropriate testbed

Ongoing Use

• Collaboration features•“A bit of smarts”• Reports for different stakeholders

Stages to deploy an IDS

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)27

Planning

IDS not only to detect attacks Management buy-in Compare different points in the

network Show economic benefit

Minimize costs, Detection efficient

Dedicated project• Involve other stakeholders• Competing priorities

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)28

Configuration and validation

Configuration hurdle Customization of the rules

Distributed environment: How to distribute alarms

How to test the IDS “All or nothing” tool

Quick tuning

Flexible criteria

Find test-bed

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)29

On-going usage

Detection of trends “A bit of smarts”

Collaboration features Incorporate changes in the systems

Better reports Meaningful reports

Artificial intelligence

Collaboration features

Flexible reports

Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)30