Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 218 times |
Download: | 1 times |
Usenix Security 2004
AutographToward Automated, Distributed Worm Signature Detection
Hyang-Ah Kim Brad KarpCarnegie Mellon University Intel Research &
Carnegie Mellon University
Usenix Security 2004 2
Internet Worm Quarantine Internet Worm Quarantine Techniques
Destination port blocking Infected source host IP blocking Content-based blocking
Worm Signature
Content-based blocking [Moore et al., 2003]
05:45:31.912454 90.196.22.196.1716 > 209.78.235.128.80: . 0:1460(1460) ack 1 win 8760 (DF)0x0000 4500 05dc 84af 4000 6f06 5315 5ac4 16c4 [email protected] d14e eb80 06b4 0050 5e86 fe57 440b 7c3b .N.....P^..WD.|;0x0020 5010 2238 6c8f 0000 4745 5420 2f64 6566 P."8l...GET./def0x0030 6175 6c74 2e69 6461 3f58 5858 5858 5858 ault.ida?XXXXXXX0x0040 5858 5858 5858 5858 5858 5858 5858 5858 XXXXXXXXXXXXXXXX . . . . .0x00e0 5858 5858 5858 5858 5858 5858 5858 5858 XXXXXXXXXXXXXXXX0x00f0 5858 5858 5858 5858 5858 5858 5858 5858 XXXXXXXXXXXXXXXX0x0100 5858 5858 5858 5858 5858 5858 5858 5858 XXXXXXXXXXXXXXXX0x0110 5858 5858 5858 5858 5825 7539 3039 3025 XXXXXXXXX%u9090%0x01a0 303d 6120 4854 5450 2f31 2e30 0d0a 436f 0=a.HTTP/1.0..Co .
Signature for CodeRed II
Signature: A Payload Content String Specific To A Worm
Usenix Security 2004 3
Content-based Blocking
Our networkX
Traffic Filtering
Internet
Signature for CodeRed II
Can be used by Bro, Snort, Cisco’s NBAR, ...
Usenix Security 2004 4
Signature derivation is too slow Current Signature Derivation Process
New worm outbreak Report of anomalies from people via phone/email/newsg
roup Worm trace is captured Manual analysis by security experts Signature generation
Labor-intensive, Human-mediated
Usenix Security 2004 5
Goal
Automatically generate signatures of previou
sly unknown Internet worms
as accurately as possible
as quickly as possible
Content-Based Analysis
Automation, Distributed Monitoring
Usenix Security 2004 6
Assumptions We focus on TCP worms that propagate
via scanning
Actually, any transport in which spoofed sources cannot communicate
successfully in which transport framing is known to monitor
Worm’s payloads share a common substring Vulnerability exploit part is not easily mutable
Not polymorphic
Usenix Security 2004 7
Outline Problem and Motivation Automated Signature Detection
Desiderata Technique Evaluation
Distributed Signature Detection Tattler Evaluation
Related Work Conclusion
Usenix Security 2004 8
Desiderata Automation: Minimal manual intervention
Signature quality: Sensitive & specific Sensitive: match all worms low false negative
rate Specific: match only worms low false positive
rate
Timeliness: Early detection
Application neutrality Broad applicability
Usenix Security 2004 9
Automated Signature Generation
Step 1: Select suspicious flows using heuristics Step 2: Generate signature using content-
prevalence analysis
Our network
Traffic Filtering
Internet Autograph Monitor
Signature
X
SignatureSignature
Usenix Security 2004 10
Heuristic: Flows from scanners are suspicious Focus on the successful flows from IPs who made unsuccessful con
nections to more than s destinations for last 24hours Suitable heuristic for TCP worm that scans network
Suspicious Flow Pool Holds reassembled, suspicious flows captured during the last time p
eriod t Triggers signature generation if there are more than flows
S1: Suspicious Flow SelectionReduce the work by filtering out vast amount of innocuous flows
Autograph (s = 2)
Non-existent
Non-existentThis flow will be
selected
Usenix Security 2004 11
S1: Suspicious Flow Selection
Heuristic: Flows from scanners are suspicious Focus on the successful flows from IPs who made unsuccessful con
nections to more than s destinations for last 24hours Suitable heuristic for TCP worm that scans network
Suspicious Flow Pool Holds reassembled, suspicious flows captured during the last time p
eriod t Triggers signature generation if there are more than flows
Reduce the work by filtering out vast amount of innocuous flows
Usenix Security 2004 12
S2: Signature Generation
All instances of a worm have a common byte pattern specific to the worm
Rationale Worms propagate by duplicating themselves Worms propagate using vulnerability of a service
Use the most frequent byte sequences across suspicious flows as signatures
How to find the most frequent byte sequences?
Usenix Security 2004 13
Worm-specific Pattern Detection Use the entire payload
Brittle to byte insertion, deletion, reordering
GARBAGEEABCDEFGHIJKABCDXXXXFlow 1
Flow 2 GARBAGEABCDEFGHIJKABCDXXXXX
Usenix Security 2004 14
Worm-specific Pattern Detection
Partition flows into non-overlapping small blocks and count the number of occurrences
Fixed-length Partition Still brittle to byte insertion, deletion, reordering
GARBAGEEABCDEFGHIJKABCDXXXXFlow 1
Flow 2 GARBAGEABCDEFGHIJKABCDXXXXX
Usenix Security 2004 15
Worm-specific Pattern Detection Content-based Payload Partitioning (COPP)
Partition if Rabin fingerprint of a sliding window matches Breakmark Configurable parameters: content block size (minimum, average, ma
ximum), breakmark, sliding window Content Blocks
Breakmark = last 8 bits of fingerprint (ABCD)
GARBAGEEABCDEFGHIJKABCDXXXXFlow 1
Flow 2 GARBAGEABCDEFGHIJKABCDXXXXX
Usenix Security 2004 16
Why Prevalence?
Worm flows dominate in the suspicious flow pool Content-blocks from worms are highly ranked
Nimda
CodeRed2
Nimda (16 different payloads)
WebDAV exploit
Innocuous, misclassified
Prevalence Distribution in Suspicious Flow Pool - From 24-hr http traffic trace
Usenix Security 2004 17
Select Most Frequent Content Block
A B D
A B E
A C E
A D
C F
C D G
B
f0
f1
f2
f3
f4
f5
H I Jf6
I H Jf7
G I Jf8
Usenix Security 2004 18
A
A
A
E
E
A
FC
C
C
D
D
DB
B
B H
H
G
G
I
I
I
J
J
J
Select Most Frequent Content Block
D
C
E
E
A
A
A
A D
FC
C D G
B
B
B
H
H
G
I
I
I
J
J
J
f0
f1
f2
f3
f4
f5
f6
f7
f8
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I J
Usenix Security 2004 19
Select Most Frequent Content Block
A
B
D
A
B E
A
C
E
A
D
C
F
C
D
GB H
I J
I
H
J
GI J
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I JP≥3
W≥90%Signature:
W: target coverage in suspicious flow poolP: minimum occurrence to be selected
Usenix Security 2004 20
Signature: A
Select Most Frequent Content Block
A
B
D
A
B E
A
C
E
A
D
C
F
C
D
GB H
I J
I
H
J
GI J
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I JP≥3
W≥90%
W: target coverage in suspicious flow poolP: minimum occurrence to be selected
Usenix Security 2004 21
Select Most Frequent Content Block
B
DBA
A
A
C E
E
A
D
F
C
C
D
GB H
I J
I
H
J
GI J
P≥3
W≥90%Signature: A
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I J
W: target coverage in suspicious flow poolP: minimum occurrence to be selected
Usenix Security 2004 22
Select Most Frequent Content Block
F
C
C D
G H
I J
I
H
J
GI J
P≥3
W≥90%Signature: A
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I J
I
W: target coverage in suspicious flow poolP: minimum occurrence to be selected
Usenix Security 2004 23
Select Most Frequent Content Block
F
C
C DG
P≥3
W≥90%Signature: A
f0 C F
f1 C D G
f2 A B D
f3 A C E
f4 A B E
f5 A B D
f6 H I J
f7 I H J
f8 G I J
ISignature:
W: target coverage in suspicious flow poolP: minimum occurrence to be selected
Usenix Security 2004 24
Outline Problem and Motivation Automated Signature Detection
Desiderata Technique Evaluation
Distributed Signature Detection Tattler Evaluation
Related Work Conclusion
Usenix Security 2004 25
Behavior of Signature Generation
Objectives Effect of COPP parameters on signature quality
Metrics Sensitivity = # of true alarms / total # of worm
flows false negatives Efficiency = # of true alarms / # of alarms
false positives Trace
Contains 24-hour http traffic Includes 17 different types of worm payloads
Usenix Security 2004 26
Signature Quality
Larger block sizes generate more specific signatures A range of w (90-95%, workload dependent)
produces a good signature
Usenix Security 2004 27
Outline Problem and Motivation Automated Signature Detection
Desiderata Technique Evaluation
Distributed Signature Detection Tattler Evaluation
Related Work Conclusion
Usenix Security 2004 28
Signature Generation Speed Bounded by worm payload accumulation speed
Aggressiveness of scanner detection heuristics: # of failed connection peers to detect a scanner
# of payloads enough for content analysis: suspicious flow pool size to trigger signature generation
Single Autograph Worm payload accumulation is slow
InternetInternet
A
AA
A
A A
A
tattler
Distributed Autograph Share scanner IP list Tattler: limit bandwidth
consumption within a predefined cap
Usenix Security 2004 29
Benefit from tattler Worm payload accumulation (time to catch 5 worms)
Signature generation More aggressive scanner detection (s) and signature
generation trigger () faster signature generation, more false positives
With s=2 and =15, Autograph generates the good worm signature before < 2% hosts get infected
Info Sharing
Autograph Monitor
Fraction of Infected Hosts
Aggressive(s = 1)
Conservative (s = 4)
NoneLuckiest 2% 60%Median 25% --
Tattler All <1% 15%
Many innocuous misclassified flows
Usenix Security 2004 30
Related Work Automated Worm Signature Detection
Distributed Monitoring Honeyd[Provos2003], DOMINO[Yegneswaran et al. 2004] Corroborate faster accumulation of worm payloads/scanner IPs
EarlyBird[Singh et al. 2003]
HoneyComb[Kreibich et al. 2003]
Autograph
Signature Generation
Content prevalence
Address Dispersion
Honeypot + Pairwise LCS
Suspicious flow selection
Content prevalence
Deployment
Network Host Network
Flow Reassembly
No Yes Yes
Distributed Monitoring
No No Yes
Usenix Security 2004 31
Future Work Attacks
Overload Autograph Abuse Autograph for DoS attacks
Online evaluation with diverse traces & deployment on distributed sites
Broader set of suspicious flow selection heuristics Non-scanning worms (ex. hit-list worms, topological worms, email w
orms) UDP worms
Egress detection Distributed agreement for signature quality testing
Trusted aggregation
Usenix Security 2004 32
Conclusion Stopping spread of novel worms requires
early generation of signatures Autograph: automated signature detection
system Automated suspicious flow selection→ Automated
content prevalence analysis COPP: robustness against payload variability Distributed monitoring: faster signature
generation Autograph finds sensitive & specific
signatures early in real network traces
Usenix Security 2004
For more information, visit
http://www.cs.cmu.edu/~hakim/autograph
Usenix Security 2004 34
Attacks Overload due to flow reassembly
Solutions Multiple instances of Autograph on separate HW (port-disjoint) Suspicious flow sampling under heavy load
Abuse Autograph for DoS: pollute suspicious flow pool
Port scan and then send innocuous trafficSolution Distributed verification of signatures at many monitors
Source-address-spoofed port scanSolution Reply with SYN/ACK on behalf of non-existent hosts/services
Usenix Security 2004 35
Number of Signatures
Smaller block sizes generate small # of signatures
Usenix Security 2004 36
tattler
A modified RTCP (RTP Control Protocol) Limit the total bandwidth of announcements sent to
the group within a predetermined cap
Usenix Security 2004 37
Simulation Setup
About 340,000 vulnerable hosts from about 6400 ASes Took small size edge networks (/16s) based on BGP table o
f 19th of July, 2001. Service deployment
50% of address space within the vulnerable ASes is reachable 25% of reachable hosts run web server 340,000 vulnerable hosts are randomly placed.
Scanning 10probes per second Scanning the entire non-class-D IP address space
Network/processing delays Randomly chosen in [0.5, 1.5] seconds