Network Security Today: Finding Complex Attacks at 100Gb/s · Sabotage. Network Security Today...

Post on 01-Jan-2021

2 views 0 download

transcript

Network Security Today

Robin Sommer!International Computer Science Institute, &!

Lawrence Berkeley National Laboratory

robin@icsi.berkeley.edu http://www.icir.org/robin

Network Security Today: Finding Complex Attacks at 100Gb/s

Network Security Today

The Old Days …

2

Border Traffic!Lawrence Berkeley National Lab (Today)!

10GE upstream, 4,000 user, 12,000 hosts

Total connections

Network Security Today

The Old Days …

2

Border Traffic!Lawrence Berkeley National Lab (Today)!

10GE upstream, 4,000 user, 12,000 hosts

Attempted connectionsSuccessful connectionsTotal connections

Network Security Today

The Old Days …

2

Border Traffic!Lawrence Berkeley National Lab (Today)!

10GE upstream, 4,000 user, 12,000 hosts

Attempted connectionsSuccessful connectionsTotal connections

Network Security Today

Today’s Threats

3

Network Security Today

Today’s Threats

3

Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.

Source: Gary Larson

Network Security Today

Today’s Threats

3

Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.

Trend 2: High-skill / high-resource attacks.!Activist Hacking.!Advanced Persistent Threats / Nation-states.

Source: Wikimedia CommonsSource: Computer Security Articles Source: EFF

Network Security Today

Today’s Threats

3

Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.

Trend 2: High-skill / high-resource attacks.!Activist Hacking.!Advanced Persistent Threats / Nation-states.

Trend 3: Insider Attacks!Exfiltration !Sabotage

Network Security Today

Defender Challenges

Varying threat models.!No ring rules them all.

4

Network Security Today

Defender Challenges

Varying threat models.!No ring rules them all.

Semantic complexity.!The action is really at the application-layer.

4

Network Security Today

Defender Challenges

Varying threat models.!No ring rules them all.

Semantic complexity.!The action is really at the application-layer.

Volume and variability.!Network traffic is an enormous haystack.

4

Network Security Today

Deep Packet Inspection at High Speed

5

Network Security Today

Analyzing Semantics

6

Network Security Today

Analyzing Semantics

6

Tap

Internet Internal

Network

IDS

Example: Finding downloads of known malware. !

Network Security Today

Analyzing Semantics

6

Tap

Internet Internal

Network

IDS

1. Find and parse all Web traffic.!2. Find and extract binaries.!3. Compute hash and compare with database.!4. Report, and potentially kill, if found.

Example: Finding downloads of known malware. !

Network Security Today

Back in 2005 …

7

Data: Leibniz-Rechenzentrum, München

020

4060

80

TByt

es/m

onth

1997 1998 1999 2000 2001 2002 2003 2004 2005

Total bytesIncoming bytes

Total upstream bytesIncoming bytes

Munich Scientific Network (2005)!3 major universities, 1 GE upstream!~100,000 Users!~50,000 Hosts

Network Security Today

Back in 2005 …

8

Data: Leibniz-Rechenzentrum, München

050

010

0015

00

TByt

es/m

onth

1996 1998 2000 2002 2004 2006 2008 2010 2012

Total bytesIncoming bytes

Oct 2005

Total upstream bytesIncoming bytes

Munich Scientific Network (Today)!3 major universities, 2x10GE upstream!~100,000 Users!~65,000 Hosts

Network Security Today

Traditional Gap: Research vs. Operations

Conceptually simple tasks can be hard in practice.!Academic research often neglects operational constraints.!Operations cannot leverage academic results. !

We focus on working with operations.!Close collaborations with several large sites.!Extremely fruitful for both sides.

9

Network Security Today

Research Platform: Bro

10

Network Security Today

Research Platform: Bro

10

Originally developed by Vern Paxson in 1996.!

Open-source, BSD-license, maintained at ICSI and NCSA.!

In operational use since the beginning. !

Conceptually very different from other IDS.

http://www.bro.org

Network Security Today

Architecture

11

Network

Packets

Network Security Today

Architecture

11

Network

Event EngineProtocol Decoding

Events

Packets

Network Security Today

Architecture

11

Network

Event EngineProtocol Decoding

Script InterpreterAnalysis Logic

Logs

Events

Packets

Notification

Network Security Today

Architecture

11

Network

Event EngineProtocol Decoding

Script InterpreterAnalysis Logic

Logs

Events

Packets

Notification“User Interface”

Network Security Today

Script Example: Matching URLs

12

Task: Report all Web requests for a file “passwd”

Network Security Today

Script Example: Matching URLs

12

Task: Report all Web requests for a file “passwd”

!event http_request(c: connection, # Connection.! method: string, # HTTP method.! original_URI: string, # Requested URL.! unescaped_URI: string, # Decoded URL.! version: string) # HTTP version.!{! if ( method == "GET" && unescaped_URI == /.*passwd/! )! NOTICE(...); # Alarm.!}

Network Security Today

Script Example: Scan Detector

13

Task: Count failed connection attempts per source address.

Network Security Today

Script Example: Scan Detector

13

Task: Count failed connection attempts per source address.

global attempts: table[addr] of count &default=0;!!event connection_rejected(c: connection)!{! local orig = c$id$orig_h; # Get originator address.!! local n = ++attempts[orig]; # Increase counter.! ! if ( n == SOME_THRESHOLD ) # Check for threshold.! NOTICE(...); # Alarm.!}

Network Security Today

“Who’s Using It?”

14

Diverse Deployment Base Universities

Research Labs Supercomputer Centers

Government Organizations Fortune 20 Enterprises

Recent User Meetings Bro Workshops 2011/13 at NCSA

Bro Exchange 2012 at NCAR

Attended by about 50-80 operators from from 30-40 organizations

Examples Lawrence Berkeley National Lab

National Center for Supercomputing Applications National Center for Atmospheric Research

Indiana University !

... and many more sites

Fully integrated into Security Onion Popular security-oriented Linux distribution

Network Security Today

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

Bro History

1995 20101996 2012

Vern writes 1st line of code!

2013

Network Security Today

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

Bro History

1995 20101996 2012

Vern writes 1st line of code!

2013

Bro SDCI!

v2.0!New Scripts

v0.2!1st CHANGES!

entry!

v0.6!RegExps!

Login analysis!!

v0.8aX/0.9aXSSL/SMB!

STABLE releases!BroLite

v1.1/v1.2!when Stmt!Resource

tuning!Broccoli!

DPD!

v1.5!BroControl!

v0.7a90!Profiling!

State Mgmt

v1.4!DHCP/BitTorrent!

HTTP entities!NetFlow!

Bro Lite Deprecated!

v1.0!BinPAC!

IRC/RPC analyzers!64-bit support!Sane version

numbers!

v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!

v0.7a175/0.8aX !Signatures!

SMTP!IPv6 support!User manual!!

v0.7a48!Consistent CHANGES

v1.3!Ctor expressions!

GeoIP!Conn Compressor

0.8a37!Communication!

Persistence!Namespaces!Log Rotation

LBNL starts using Bro!

operationally

v2.1!IPv6!

Input Framew.

v2.2!File Analysis!

Summary Stat.

Bro Center!

Network Security Today

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

Bro History

1995

USENIX Paper!Stepping Stone

Detector!

AnonymizerActive Mapping!Context Signat.!

TRWState Mgmt.!

Independ. State!

Host Context!Time Machine!

Enterprise Traffic

BinPAC!DPD!

2nd Path

Bro ClusterShunt

Autotuning

Parallel Prototype

20101996

Academic Publications

Input Framework

2012

Vern writes 1st line of code!

2013

Bro SDCI!

v2.0!New Scripts

v0.2!1st CHANGES!

entry!

v0.6!RegExps!

Login analysis!!

v0.8aX/0.9aXSSL/SMB!

STABLE releases!BroLite

v1.1/v1.2!when Stmt!Resource

tuning!Broccoli!

DPD!

v1.5!BroControl!

v0.7a90!Profiling!

State Mgmt

v1.4!DHCP/BitTorrent!

HTTP entities!NetFlow!

Bro Lite Deprecated!

v1.0!BinPAC!

IRC/RPC analyzers!64-bit support!Sane version

numbers!

v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!

v0.7a175/0.8aX !Signatures!

SMTP!IPv6 support!User manual!!

v0.7a48!Consistent CHANGES

v1.3!Ctor expressions!

GeoIP!Conn Compressor

0.8a37!Communication!

Persistence!Namespaces!Log Rotation

LBNL starts using Bro!

operationally

v2.1!IPv6!

Input Framew.

v2.2!File Analysis!

Summary Stat.

Bro Center!

Network Security Today

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

Bro History

1995

USENIX Paper!Stepping Stone

Detector!

AnonymizerActive Mapping!Context Signat.!

TRWState Mgmt.!

Independ. State!

Host Context!Time Machine!

Enterprise Traffic

BinPAC!DPD!

2nd Path

Bro ClusterShunt

Autotuning

Parallel Prototype

20101996

Academic Publications

Input Framework

2012

Vern writes 1st line of code!

2013

Bro SDCI!

v2.0!New Scripts

v0.2!1st CHANGES!

entry!

v0.6!RegExps!

Login analysis!!

v0.8aX/0.9aXSSL/SMB!

STABLE releases!BroLite

v1.1/v1.2!when Stmt!Resource

tuning!Broccoli!

DPD!

v1.5!BroControl!

v0.7a90!Profiling!

State Mgmt

v1.4!DHCP/BitTorrent!

HTTP entities!NetFlow!

Bro Lite Deprecated!

v1.0!BinPAC!

IRC/RPC analyzers!64-bit support!Sane version

numbers!

v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!

v0.7a175/0.8aX !Signatures!

SMTP!IPv6 support!User manual!!

v0.7a48!Consistent CHANGES

v1.3!Ctor expressions!

GeoIP!Conn Compressor

0.8a37!Communication!

Persistence!Namespaces!Log Rotation

LBNL starts using Bro!

operationally

v2.1!IPv6!

Input Framew.

v2.2!File Analysis!

Summary Stat.

Bro Center!

Example: Processing performance!LBNL operations had trouble keeping up.!Research question: How can Bro scale up?

Network Security Today

Load-balancing Architecture

16

Network Security Today

Load-balancing Architecture

16

Detection LogicPacket Analysis

NIDS

10G

Network Security Today

Load-balancing Architecture

16

10G

Exte

rnal

Pac

ket L

oad-

Bala

ncer!

Flows

Detection Logic

Packet Analysis

NIDS 2

Detection Logic

Packet Analysis

NIDS 1

Detection Logic

Packet Analysis

NIDS 3

1G

1G

1G

Network Security Today

Load-balancing Architecture

16

10G

Exte

rnal

Pac

ket L

oad-

Bala

ncer!

Flows

Detection Logic

Packet Analysis

NIDS 2

Detection Logic

Packet Analysis

NIDS 1

Detection Logic

Packet Analysis

NIDS 3

Communication

Communication

1G

1G

1G

Network Security Today

Load-balancing Architecture

16

10G

Exte

rnal

Pac

ket L

oad-

Bala

ncer!

Flows

“Bro Cluster”

Detection Logic

Packet Analysis

NIDS 2

Detection Logic

Packet Analysis

NIDS 1

Detection Logic

Packet Analysis

NIDS 3

Communication

Communication

1G

1G

1G

Network Security Today

A Production Load-Balancer

1717

Network Security Today

A Production Load-Balancer

1717

cFlow: 10GE line-rate, stand-alone load-balancer

10 Gb/s in/out!Web & CLI!

Filtering capabilities!!

Network Security Today

A Production Load-Balancer

1717

cFlow: 10GE line-rate, stand-alone load-balancer

10 Gb/s in/out!Web & CLI!

Filtering capabilities!!

Network Security Today

Next Stop: 100 Gb/s

18

Source: ESNet

Now these sites need a monitoring solution ... Working with cPacket on a 100GE load-balancer!

DOE/ESNet !100G Advanced Networking Initiative

2011

Source: ESNet

Network Security Today

Next Stop: 100 Gb/s

19

Source: ESNet

2014

Network Security Today

On Deck: 400G Connectivity

20

Computational Research and Theory Building.

Oakland Scientific Facility.

100G

2 x 100G

File System Links

Inter-site Traffic

100G WAN 100G WAN

Berkeley National Laboratory

Sources: ESNet/LBNL/NERSC

Network Security Today

10G 10G10G

Science DMZ

21

Campus LAN

Internet

Network Security Today

100G 100G100G

Science DMZ

21

Campus LAN

Internet

Network Security Today

10G 10G

Science DMZ

21

Campus LAN

100GInternet

Network Security Today

10G 10G

Science DMZ

21

Campus LAN

100G

100G

Transfer/Storage Nodes

100G

Science DMZ Switch

Internet

Network Security Today

10G 10G

Science DMZ

21

Campus LAN

100G

Clean, high-bandwith path

Low-bandwidth!campus access

100G

Transfer/Storage Nodes

100G

Science DMZ Switch

Internet

Network Security Today

10G 10G10G

100G

Science DMZ

22

Campus LAN

100G

Transfer/Storage Nodes

100G

Science DMZ Switch

100GInternet

Network Security Today

100G

10G 10G10G

100G

Science DMZ

22

Campus LAN

100G

Transfer/Storage Nodes

100G

Science DMZ Switch

100GInternet

Network Security Today

100G Bro Cluster

23

100G

Science DMZ Switch

Network Security Today

100G Bro Cluster

23

100G Load-balancer

100G

Science DMZ Switch

Network Security Today

100G Bro Cluster

23

100G Load-balancer

10G

100G

Science DMZ Switch

Network Security Today

100G Bro Cluster

23

100G Load-balancer

10G

Bro Cluster

100G

Science DMZ Switch

Network Security Today

100G Bro Cluster

23

100G Load-balancer

10G

Bro Cluster

API

Con

trol

100G

Science DMZ Switch

Network Security Today

100G Bro Cluster

23

100G Load-balancer

10G

Bro Cluster

API

Con

trol

100G

Science DMZ Switch

Con

trol

API

Network Security Today

100G Bro Cluster

23

100G Load-balancer

10G

Bro Cluster

API

Con

trol

100G

Science DMZ Switch

Con

trol

API

Network Security Today

Parallelizing DPI on Multi-core Systems

24

Network Security Today

Going Multi-Core …

Bro is single-threaded!Cluster backends have muitple cores, mostly idle.!Work-around: “Cluster in a box”!

We really want multi-threading, though.!Needs to scale well with increasing numbers of cores.!Needs to be transparent to the operator.!

For some IDS, that’s not so hard.!For others, it is ...

25

Network Security Today

Concurrent Analysis

26

Network

Event EngineProtocol Decoding

Script InterpreterAnalysis Logic

Logs

Events

Packets

Notification

Network Security Today

Concurrent Analysis

26

Single Thread

Network

Event EngineProtocol Decoding

Script InterpreterAnalysis Logic

Logs

Events

Packets

Notification

Network Security Today

Concurrent Analysis

27

Event Engine

Network

Packets

Events

Notification

Script ThreadsScripting Language

Event Engine! ThreadsPacket Analysis

Detection Logic

Dispatcher Kernel or NIC

Network Security Today

Concurrent Analysis

27

Event Engine

Network

Packets

Events

Notification

Script ThreadsScripting Language

Event Engine! Threads

“Cluster in a Box”

Packet Analysis

Detection Logic

Dispatcher Kernel or NIC

Network Security Today

Concurrent Analysis

27

Event Engine

Network

Packets

Events

Notification

Script ThreadsScripting Language

Event Engine! Threads

“Cluster in a Box”

Packet Analysis

Detection Logic

Dispatcher Kernel or NIC

How to parallelize!a scripting language?

Network Security Today

How to Parallelize Event Handlers?

28

Simple: State-less Analysis

Network Security Today

How to Parallelize Event Handlers?

28

Simple: State-less Analysis

!event http_request(c: connection, # Connection.! method: string, # HTTP method.! original_URI: string, # Requested URL.! unescaped_URI: string, # Decoded URL.! version: string) # HTTP version.!{! if ( method == "GET" && unescaped_URI == /.*passwd/! )! NOTICE(...); # Alarm.!}

Network Security Today

How to Parallelize Event Handlers? (2)

29

Challenging: Analysis that keeps global state.

Network Security Today

How to Parallelize Event Handlers? (2)

29

Challenging: Analysis that keeps global state.

global attempts: table[addr] of count &default=0;!!event connection_rejected(c: connection)!{! local orig = c$id$orig_h; # Get originator address.!! local n = ++attempts[orig]; # Increase counter.! ! if ( n == SOME_THRESHOLD ) # Check for threshold.! NOTICE(...); # Alarm.!}

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

LOCK(attempts)!

++attempts[s]!

UNLOCK(attempts)!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

LOCK(attempts)!

++attempts[s]!

UNLOCK(attempts)!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

LOCK(attempts)!

++attempts[s]!

UNLOCK(attempts)!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

attempts_1

attempts_2

attempts_3

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

attempts_1

attempts_2

attempts_3

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

hash(addr)1

1

2

2

3

3

hash: addr -> {1, 2 ,3}

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

attempts_1

attempts_2

attempts_3

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

hash(addr)1

1

2

2

3

3

hash: addr -> {1, 2 ,3}

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

attempts_1

attempts_2

attempts_3

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

hash(addr)1

1

2

2

3

3

hash: addr -> {1, 2 ,3}

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

attempts_1

attempts_2

attempts_3

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

hash(addr)1

1

2

2

3

3

hash: addr -> {1, 2 ,3}

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Thread 1’s attempts

Thread 2’s attempts

Thread 3’s attempts

Network Security Today

!!

connection_rejected(c):!

!

s = c.originator !

! ++attempts[s]!

!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

! ++attempts[s]!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator!

!

++attempts_1[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_3[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_2[s]!

!

hash(addr)1

1

2

2

3

3

hash: addr -> {1, 2 ,3}

Thread 1!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread 3!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread 2!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts_(hash(s))[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 134.96.7.179 !

s = c.originator !

! ++attempts[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 131.159.15.49 !

s = c.originator !

!

++attempts[s]!

!

Thread hash(s)!!

connection_rejected(c):!

# 192.150.187.12!

s = c.originator !

!

++attempts[s]!

!

Parallelizing Event Execution

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

30

attempts[addr] of count

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

addr count131.234.142.33 12

134.96.7.179 32

141.142.192.147 71

192.150.187.12 8

128.3.41.105 555

131.159.15.49 1

Thread 1’s attempts

Thread 2’s attempts

Thread 3’s attempts

Network Security Today

Parallel Event Scheduling

31

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

http_request

Conn Y

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

http_reply

Conn

http_request

Conn Y

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

http_reply

Conn

http_request

Conn Y

http_reply

Conn Y

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

http_reply

Conn

http_request

Conn Y

http_reply

Conn Y

conn_rejected

Orig A

conn_rejected

Orig A

Network Security Today

Parallel Event Scheduling

31

Thread!1

Thread!2

Thread!3

Thread!4 … Thread!

n

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

Que

ue

conn_rejected

Orig A

conn_rejected

Orig B

http_request

Conn X

http_reply

Conn

http_request

Conn Y

http_reply

Conn Y

conn_rejected

Orig A

conn_rejected

Orig A

Challenge: Implementing this …

Network Security Today

New Platform: Abstract Machine

32

A High-Level Intermediary Language for Traffic Inspection

Network Security Today

New Platform: Abstract Machine

32

First-class networking types

built-in

Containers with state management

support

Platform for building high-level, reusable

functionality onDomain-specific

concurrency modelWell-defined,

contained execution environment

Domain-specific Data Types

Robust/Secure Execution

Concurrent Analysis

High-level Standard

Components

State Management

Timers can drive execution

Real-time Performance

Support for incremental processing

Extensive optimization

potential

Scalability through parallelization

Static type-system, and robust error

handlingCompilation to

native code

A High-Level Intermediary Language for Traffic Inspection

Network Security Today

New Platform: Abstract Machine

32

First-class networking types

built-in

Containers with state management

support

Platform for building high-level, reusable

functionality onDomain-specific

concurrency modelWell-defined,

contained execution environment

Domain-specific Data Types

Robust/Secure Execution

Concurrent Analysis

High-level Standard

Components

State Management

Timers can drive execution

Real-time Performance

Support for incremental processing

Extensive optimization

potential

Scalability through parallelization

Static type-system, and robust error

handlingCompilation to

native code

A High-Level Intermediary Language for Traffic Inspection

Network Security Today

Summary

33

Network Security Today

Conclusions

Threats have changed.!Detection requires deep, flexible, semantic analysis.!

Working to push the limits. !Leverage capabilities of modern network hardware.!Exploit parallelism inherent in network traffic analysis.!

Bro is an ideal platform for such work.!Operationally deployed across the country.!Bridges traditional gap between academia and operations. !

34

Network Security Today

Robin Sommer!International Computer Science Institute, &!

Lawrence Berkeley National Laboratory

robin@icsi.berkeley.edu http://www.icir.org/robin

Thanks for you attention!

35