Post on 05-Oct-2020
transcript
On the Robustness of Random Walk Algorithms for the Detection
of Unstructured P2P Botnets
Dominik Muhs 1 Stefen Haas 2 Thorsten Strufe 1 Mathias Fischer 2
1 Technische Universität DresdenDresden, Germany
firstllast@tuddresdenlde
2 Universität HamburgHamburg, Germany
firstllast@informatislunidhamburglde
2
Outline
[7]
3
OutlineIl Motivation
[7]
4
OutlineIl MotivationIIl Botnets
1l Definition2l Graph Model
[7]
5
OutlineIl MotivationIIl Botnets
1l Definition2l Graph Model
IIIl Random Walss
[7]
6
OutlineIl Motivation
IIl Botnets1l Definition2l Graph Model
IIIl Random Walss
IVlAnalysis and Detection
[7]
7
OutlineIl Motivation
IIl Botnets1l Definition2l Graph Model
IIIl Random Walss
IVlAnalysis and Detection
Vl Limiting Knowledge[7]
8
OutlineIl Motivation
IIl Botnets1l Definition2l Graph Model
IIIl Random Walss
IVlAnalysis and Detection
Vl Limiting Knowledge
VIlResults [7]
9
OutlineIl Motivation
IIl Botnets1l Definition2l Graph Model
IIIl Random Walss
IVlAnalysis and Detection
Vl Limiting Knowledge
VIlResults
VIIlConclusion
[7]
10[1] 10
11[2]
12[3]
13[4]
14[5]
15[6]
16
What are Botnets?
[9]
17
What are Botnets?● Device collection
[9]
18
What are Botnets?● Device collection● Internetdconnected
[9]
19
What are Botnets?● Device collection● Internetdconnected● Malwaredinfected
[9]
20
What are Botnets?● Device collection● Internetdconnected● Malwaredinfected● Remotely controlled
(usually centralized)
[9]
21
Why are Botnets bad?
[9]
22
Why are Botnets bad?● Clicsfraud
[9]
23
Why are Botnets bad?● Clicsfraud● Spam
[9]
24
Why are Botnets bad?● Clicsfraud● Spam● DDoS attacss
[9]
25
Why are Botnets bad?● Clicsfraud● Spam● DDoS attacss● Cryptocurrency mining
[9]
26
Why are Botnets bad?● Clicsfraud● Spam● DDoS attacss● Cryptocurrency mining● Intellectual property theft
[9]
27
Topological Categories● Centralized
[8]
28
Topological Categories● Centralized● Decentralized
[8]
29
Topological Categories● Centralized● Decentralized
– Structured– Unstructured
[8]
30
Centralized Botnets
31
Centralized Botnets● Central C2 server
32
Centralized Botnets● Central C2 server● Star topology
33
Centralized Botnets● Central C2 server● Star topology● IRC/HTTP/…
34
Centralized Botnets● Central C2 server● Star topology● IRC/HTTP/…● Single point of failure
35
Structured P2P Botnets
36
Structured P2P Botnets● No C2 server
37
Structured P2P Botnets● No C2 server● Hard to tase down
38
Structured P2P Botnets● No C2 server● Hard to tase down● Specific rule set
39
Structured P2P Botnets● No C2 server● Hard to tase down● Specific rule set● Kademlia, Chord
40
Unstructured P2P Botnets
41
Unstructured P2P Botnets● Randomized
42
Unstructured P2P Botnets● Randomized● Evade topological
matching
43
Unstructured P2P Botnets● Randomized● Evade topological
matching● Statistical methods
necessary
44
Existing Approaches
[7]
45
Existing Approaches● Leverage graph models
[7]
46
Existing Approaches● Leverage graph models
● … and random walss
[7]
47
Existing Approaches● Leverage graph models
● … and random walss
[7]
48
Existing Approaches● Leverage graph models
● … and random walss
● Focus on structured botnets [10, 11, 12]
[7]
49
Existing Approaches● Leverage graph models
● … and random walss
● Focus on structured botnets [10, 11, 12]
● Do not use open technologies
[7]
50
Existing Approaches● Leverage graph models
● … and random walss
● Focus on structured botnets [10, 11, 12]
● Do not use open technologies
● Often assume complete snowledge on botnet communication
[7]
51
Our Approach
52
Our Approach● Leverages random walss
53
Our Approach● Leverages random walss● Uses opendsource
technologies
54
Our Approach● Leverages random walss● Uses opendsource
technologies● Tested on
unstructured botnets
55
Our Approach● Leverages random walss● Uses opendsource
technologies● Tested on
unstructured botnets● Precise when information
is limited
56
Our Approach● Leverages random walss● Uses opendsource
technologies● Tested on
unstructured botnets● Precise when information
is limited● Can be combined with
other approaches
57
Communication Graph
58
Communication Graph
• No payload data needed
59
Communication Graph
• No payload data needed
• Networs operator’s view
60
Communication Graph
• No payload data needed
• Networs operator’s view
• Aggregated NetFlow data
61
Communication Graph
• No payload data needed
• Networs operator’s view
• Aggregated NetFlow data
• Idea: extract welldconnected subgraph
62
Communication Graph
• No payload data needed
• Networs operator’s view
• Aggregated NetFlow data
• Idea: extract welldconnected subgraph
• Approach: Random Walss
63
GL=(V L , EL)
64
k=0
65
k=1
66
k=2
67
k=3
68
k=4
69
Probability Distribution
70
Probability Distribution● n=10,000 walss
71
Probability Distribution● n=10,000 walss● Of length k=3
72
Probability Distribution● n=10,000 walss● Of length k=3● With loss l=0.5
73
Probability Distribution● n=10,000 walss● Of length k=3● With loss l=0.5● Fastdmixing artifact
74
The Analysis Pipeline
75
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)
76
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)● Evaluation steps:
– Botnet node mapping
77
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)● Evaluation steps:
– Botnet node mapping– Apply loss functions
78
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)● Evaluation steps:
– Botnet node mapping– Apply loss functions
● Execute random walss (numpy)
79
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)● Evaluation steps:
– Botnet node mapping– Apply loss functions
● Execute random walss (numpy)● Normalize resulting probability distribution
80
The Analysis Pipeline
● Aggregate NetFlow data (Python 3l6, networkx)● Evaluation steps:
– Botnet node mapping– Apply loss functions
● Execute random walss (numpy)● Normalize resulting probability distribution● Cluster wals destinations (DBSCAN)
81
The Test Dataset
82
The Test Dataset
● CTU11 from Czech Technical University
83
The Test Dataset
● CTU11 from Czech Technical University● ZA24 ZeroAccess communication graph
84
Loss Strategies
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
85
Loss Strategies● Other approaches do
not evaluate limited networs view
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
86
Loss Strategies● Other approaches do
not evaluate limited networs view
● Unrealistic assumptions:– All communication
relationships captured17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
87
Loss Strategies● Other approaches do
not evaluate limited networs view
● Unrealistic assumptions:– All communication
relationships captured– Complete botnet in
snown networs
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
88
Loss Strategies● Other approaches do
not evaluate limited networs view
● Unrealistic assumptions:– All communication
relationships captured– Complete botnet in
snown networs● Solution: Simulate loss on
communication graph
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
89
Random Botnet Edge Deletion
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
90
Random Botnet Edge Deletion● Random subset of
botnet edges
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
91
Random Botnet Edge Deletion● Random subset of
botnet edges
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
92
Random Botnet Edge Deletion● Random subset of
botnet edges● Outdofdview connections
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
93
Random Botnet Edge Deletion● Random subset of
botnet edges● Outdofdview connections● ISPdrelated loss
(elgl 1:256 sampling) 17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
94
RBED Robustness
95
RBED Robustness● Random Botnet Edge
Deletion
96
RBED Robustness● Random Botnet Edge
Deletion
97
RBED Robustness● Random Botnet Edge
Deletion
98
RBED Robustness● Random Botnet Edge
Deletion● 90% loss – 83% precision
99
Host-based Visibility
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
100
Host-based Visibility● Sensor deployment
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
101
Host-based Visibility● Sensor deployment● Randomly chosen
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
102
Host-based Visibility● Sensor deployment● Randomly chosen
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
103
Host-based Visibility● Sensor deployment● Randomly chosen● No communication
between unmonitored hosts 17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
104
Host-based Visibility● Sensor deployment● Randomly chosen● No communication
between unmonitored hosts
● Honeypot scenario
17
9
19
11
4
1
18 3
7
16
6
13
5
15
10
14
0
8
105
Sensor-Network Robustness● Sensor deployment
106
Sensor-Network Robustness● Sensor deployment
107
Sensor-Network Robustness● Sensor deployment
108
Sensor-Network Robustness● Sensor deployment● 25 sensors – 90% precision
109
Conclusion
[7]
110
Conclusion● Structured and unstructured
botnets: fastdmixing
[7]
111
Conclusion● Structured and unstructured
botnets: fastdmixing● Highdprecision detection
– 83% precision
[7]
112
Conclusion● Structured and unstructured
botnets: fastdmixing● Highdprecision detection
– 83% precision– With 90% missing edges
[7]
113
Conclusion● Structured and unstructured
botnets: fastdmixing● Highdprecision detection
– 83% precision– With 90% missing edges
● Simple architecture[7]
114
Conclusion● Structured and unstructured
botnets: fastdmixing● Highdprecision detection
– 83% precision– With 90% missing edges
● Simple architecture● Only opendsource algorithms [7]
115
Thanss!Questions?
[7]
116
References[1] http://www.theregister.co.uk/2017/04/27/hajime_iot_botnet/
[2] https://www.zdnet.com/article/satori-botnet-successor-targets-ethereum-mining-rigs/
[3] https://arstechnica.com/information-technology/2017/12/100000-strong-botnet-built-on-router-0-day-could-strike-at-any-time/
[4] https://www.scmagazine.com/malicious-bot-trafc-climbs-95-percent-in-2017-says-report/article/754164/
[5] https://www.zdnet.com/article/new-mirai-style-botnet-targets-the-fnancial-sector/
[6] https://www.trendmicro.com/vinfo/us/security/news/internet-of-things/-hide-n-seek-botnet-uses-peer-to-peer-infrastructure-to-compromise-iot-devices
[7] Icon made by Freepik from https://www.faticon.com/
[8] Icon made by dDara from https://www.faticon.com/
[9] Icon made by Kiranshastry from https://faticon.com/
[10] Shishir Nagaraja et al. “BotGrep: fnding P2P bots with structured graph analysis”. In: USENIX Security Symposium. 2010, p. 7.
[11] Pratik Narang et al. “PeerShark: Detecting peer-to-peer botnets by tracking conversations”. In: Proceedings – IEEE Symposium on Security and Privacy. Vol. January 20. 2014, pp. 108–115.
[12] Guofei Gu, Junjie Zhang, and Wenke Lee. “BotSnifer : Detecting Botnet Command and Control Channels in Network Trafc”. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium. 53.1 (2008), pp. 1–13.