Netzob Tutorial

8/11/2019 Netzob Tutorial

1/162

netzob.org@Netzob

The Future of Protocol Reversing and

Simulation Applied on ZeroAccess

29C3, am!urgDecember 29 2012

@Netzob


2/162

netzob.org@Netzob

Authors""" You talkin' to me? Travis Bickle


3/162

netzob.org@Netzob

#es, $e%re French &


4/162

netzob.org@Netzob

Fr!ric "#$%&Y (@)*+gu*,

$- *ecurit+ engineer

eer*e engineering

/+*tem anal+*i* an! ar!ening

-ru*te! omuting


5/162

netzob.org@Netzob

"eorge* 34//&- (@5aeluce,

6D *tu!ent

$ntru*ion Detection

3otnet *imulation

6rotocol learning

Supelec C'(ree*earc team

7!i*er* 8 Guillaume Hiet Ludovic M


6/162

netzob.org@Netzob

74//Y/ : France

7u!it an! ealuation

$-/&F lab (ommon riteria: /6N*: ...,

6ente*t lab

;D

$$$"amos

s)s"fr


7/162

netzob.org@Netzob

Topics"""

"o aea!: make m+ !a+ Harry Callahan


8/162

netzob.org@Netzob

Reverse rotocol*Simulate en!oint*

*ap botnet*


9/162

netzob.org@Netzob

+h) reverse engineering of

protocols -


10/162

netzob.org@Netzob

6rotocol* are eer+


11/162

netzob.org@Netzob

6rotocol* are eer+


12/162

netzob.org@Netzob

7**e** te robu*tne** o= imlementation*

&> 8 Fuzz te control 76$ o= a centri=uge

29 &> 8

an+ -amagotci* ere %arme! in te

aking o= ti* 6re*entation

&A/i 3ea*t


13/162

netzob.org@Netzob

7**e** te robu*tne** o= imlementation*

&> 8 Fuzz te control 76$ o= a centri=uge

29 &> 8

an+ -amagotci* ere %arme! in te

aking o= ti* 6re*entation

&A/i 3ea*t


14/162

netzob.org@Netzob

7nal+ze tra==ic an! i!enti=+ otential

!ata leakage

&> 8 7re +ou *ure +our $6 eutation

7liance !oe*n't leak +our email* ?


15/162

netzob.org@Netzob

omare te imlementation o= a rotocol

8 ealuation* o= cr+to ro!uct*


16/162

netzob.org@Netzob

-o !eelo a =ree er*ion o= a rorietar+

imlementation

&> 8 Dre< Fi*er '* talk @ 2B on Cinect &


17/162

netzob.org@Netzob

Current reverse engineering

approach"""


18/162

netzob.org@Netzob

%ae +ou eer (trie! to,& a rotocol ?


19/162

netzob.org@Netzob

Di! it looke! like ti* ?


20/162

netzob.org@Netzob

omle>

-imecon*uming

o*tl+ anual



21/162

netzob.org@Netzob

omle>

-imecon*uming

o*tl+ anual

MOSTLYVISUA

L



22/162

netzob.org@Netzob

/oul!


23/162

netzob.org@Netzob

/et%s see if $e can/et%s see if $e can automateautomate

some R0 tas1s"""some R0 tas1s"""


24/162

netzob.org@Netzob

Some reminders a!out protocols


25/162

netzob.org@Netzob

/et%s eamine the TCP protocol


26/162

netzob.org@Netzob

/YN me**age

7C me**age

6#/% me**age F$N me**age

/- me**age

...

Di==erent t+e* o= me**age*


27/162

netzob.org@Netzob

oncet o= enca*ulation la+er*


28/162

netzob.org@Netzob

Fiel!* artitioning


29/162

netzob.org@Netzob

elation*

$ntrame**age relation*


30/162

netzob.org@Netzob

elation*

$nterme**age relation*


31/162

netzob.org@Netzob

onte>tual alue*


32/162

netzob.org@Netzob

7licationleel alue*


33/162

netzob.org@Netzob

/eEuence o= ali! me**age*


34/162

netzob.org@Netzob

/et%s find a modelthat

covers protocol attri!utes


35/162

netzob.org@Netzob

7ca!emic* are er+ goo!


36/162

netzob.org@Netzob

Design and Validation of Comuter !rotocols

b+ ". %olzmann

7 ommunication 6rotocol i* ma!e o=

G !i*tinct art* . . .


37/162

netzob.org@Netzob

a service 4567


38/162

netzob.org@Netzob

a service 4567


39/162


40/162

netzob.org@Netzob

a voca!ular)o= me**age* 3567


41/162

netzob.org@Netzob

te encoding(=ormat, o= eac me**age8567


42/162

netzob.org@Netzob

te procedure rules6567

#es, that $as an academic model


43/162

netzob.org@Netzob

,


44/162

netzob.org@Netzob

Reduced model for a Protocol

a ocabular+ a li*t o= e**age Format

a grammar /tate acine


45/162

netzob.org@Netzob

'ntroducing .eto! """'ntroducing .eto! """

: l f . t !


46/162

netzob.org@Netzob

:oals of .eto!

$n=er unkno


47/162

netzob.org@Netzob

:oals of .eto!

$n=er unkno


48/162

netzob.org@Netzob

:oals of .eto!

$n=er unkno


49/162

netzob.org@Netzob

7roac taken b+ Netzob

6a**ie an! actie in=erence

/emi7utomatic 7roac

No binar+ maniulation


50/162

netzob.org@Netzob


51/162

netzob.org@Netzob


52/162

netzob.org@Netzob

Netzob imlementation

"raical inter=ace ("-C,

o*tl+


53/162

netzob.org@Netzob

"eorge* 3o**ert

Fr!ric "uir+

"uillaume %iet

4liier -tar!

a>ime 4liier

7le>an!re 6ign

"oulen "uieu>

Frank olan!

Fabien 7n!r Juentin %e+ler

3enKamin Du=our

"iu*ee ma**aro

Netzob'* /on*or*

/tate o= te art boun!arie*


54/162

netzob.org@Netzob

/tate o= te art boun!arie*

The un1no$n

Netzob

/anguage Theor)

Reverse 0ngineering

:rammar 'nference

;otnet ;ehavioural Anal)sis

Fuing

Sum of human 1no$ledge

N& /tate o= te art boun!arie*


55/162

netzob.org@Netzob

The un1no$n

.eto!

Based on an original idea of Matt Might

.e$ sum of human 1no$ledge

/anguage Theor)

Reverse E!ieeri!

"rammar I#erece

$otet $e%avioural Aalysis

Fuing


56/162

netzob.org@Netzob

R0 Zero Access C


57/162

netzob.org@Netzob

Zero Access a1a Sirefef7

ecent botnet (/et. 2011,

*till in actiit+

LM 1 million zombie* (9 million* in*talle!,

lick =rau! an! bitcoin miner

7t lea*t 2 er*ion* o= te rootkit

#gra!e! 626 rotocol

3a*e! on /oo* an! Cin!*igt eort*



58/162

netzob.org@Netzob


ultile 626 management me**age*

6eer* !irector+ retrieal

File* !irector+ retrieal

#D6 ; -6 connection*

#D6 =or me**age* (u!81IHIH,

-6 =or !ata

%ar! co!e! 3oot*tra 6eer*

&> 8 IB.G1.10B.2HG: (,: 21I.211.1B1.22I


59/162

netzob.org@Netzob

/et%s pla) $ith its P2P protocol


60/162

netzob.org@Netzob

Re=uirements


61/162

netzob.org@Netzob

Fe< realcommunication trace*

O7cce** 8 *ome trace*


62/162

netzob.org@Netzob

7 con=ine! enironment an! te binar+

7!ate! Pirtual acine* L Fire


63/162

netzob.org@Netzob

7 con=ine! enironment an! te binar+

7!ate! Pirtual acine* L Fire


64/162

netzob.org@Netzob

Step 4 > :et messages


65/162

netzob.org@Netzob

ature dataflo$s

(Net


66/162

netzob.org@Netzob

(Net:'...SH...5......8.BT..E..!...J..........

5.J*Y.B8..m.0...!.&.....OK.R...5.F...B8.Sa!...*.....U."g....V...B8...W...o......(t0...i......oX..?...Z.Qc..Me..z......Ug$.-.*.m&..L..T

.[eoDz.W7t..!.J6./A\,.8.X#.]..t...g.P..G.2.......4..=2.AB(......r..]......./..;B...Q..

'.........^.

..g.......;(6..-...+.....G..2.................;......i..e

...._.56....N..A./P.X7........`.Y.[....TAPt!....t..

....\.4..?... Jga(R...T...8....5#..S.'.i.1\...ui.

5B.][email protected].]...2Gt.".,G.:......./......9.'.5G.X.

?..b([email protected]..(..........P...5......=.B8?.e..!..........>.F.5...u..8.BT.u..!.V......).'...5.O.

B8...H...!.b............5...k.B8W.e.!..........U."g....V...B8...W...o......(t0...i......oX..

?...Z.Qc..Me..z......Ug$.-.*.m&..L..T.[eoDz.W7t..!.J6./A\,.8.X#.]..t...g.P..G.2.......

4..=2.AB(......r..]......./..;B...Q..'.........^.

..g.......;(6..-...+.....G..2.................;......i..e

...._.56....N..A./P.X7........`.Y.[....TAPt!....t..

....\.4..?... Jga(R...T...8....5#..S.'.i.1\...ui.


/lit dataflo$s in messages

(*ub rotocol kno


67/162

netzob.org@Netzob

(*ub rotocol kno:'...SH...5......8.BT..E..!...J..........

5.J*Y.B8..m.0...!.&.....OK.R...5.F...B8.Sa!...*.....U."g....V...B8...W...o......(t0...i......oX..?...Z.Qc..Me..z......Ug$.-.*.m&..L..T

.[eoDz.W7t..!.J6./A\,.8.X#.]..t...g.P..G.2.......4..=2.AB(......r..]......./..;B...Q..

'.........^.

..g.......;(6..-...+.....G..2.................;......i..e

...._.56....N..A./P.X7........`.Y.[....TAPt!....t..

....\.4..?... Jga(R...T...8....5#..S.'.i.1\...ui.


-""!"""""""?"p"gN..(..........P...5......=.B8?.e..!..........>.F.5...u..8.BT.u..!.V......).'...5.

O.

B8...H...!.b............5...k.B8W.e.!..........U."g....V...B8...W...o......(t0...i......oX..

?...Z.Qc..Me..z......Ug$.-.*.m&..L..T.[eoDz.W7t..!.J6./A\,.8.X#.]..t...g.P..G.2.......4..=2.AB(......r..]......./..;B...Q..

'.........^...g.......;(6..-...+.....G..2.................;......i..e

...._.56....N..A./P.X7........`.Y.[....TAPt!....t..

....\.4..?... Jga(R...T...8....5#..S.'.i.1\...ui.


e**age 1

e**age 2

e**age

e**age H


68/162

netzob.org@Netzob

Netzob =rame


69/162

netzob.org@Netzob

Step 2 > R0 voca!ular)


70/162

netzob.org@Netzob

A!stract messages


71/162

netzob.org@Netzob

1 me**age \ a *orte! receie! or *ent

*eEuence o= bit*

010110101001000101010110100101011101010001010010


72/162

netzob.org@Netzob

1 me**age \ a *orte! receie! or *ent

*eEuence o= bit*

*eci= ic to a conte>t

%mails& '!s& Timestams& B'D& (dd'D& )))

010110101001000101010110100101011101010001010010


73/162

netzob.org@Netzob

e ae to !econte>tualizeme**age*

-e $D&7 8

egrou me**age* b+ *imilarit+ an! =in!

conte>tual ariation*


74/162

netzob.org@Netzob

+e consider similar messages !ased on their

commo partitioi!

e**age* are *litte! in Fields u*ing


75/162

netzob.org@Netzob



76/162

netzob.org@Netzob

/imle 7lignment

Delimitorba*e! 7lignment

/eEuence 7lignment



77/162

netzob.org@Netzob

/imle 7lignment


/eEuence 7lignment



78/162

netzob.org@Netzob

/imle 7lignment


/eEuence 7lignment

Nee!leman ; un*c

3ut -F i* Nee!leman ; un*c ?


79/162

netzob.org@Netzob

/eEuence alignment


80/162

netzob.org@Netzob

alie! to & o= rotocol* (c.=. ar*all 3e!oe,

]0 B 2= IG b! BI ]a !2 00

]0 cH 00 00

e *tart


81/162

netzob org@Netzob

e buil! a !i*tance matri>

]0 B 2= IG b! BI ]a !2 00

]0

cH

00

00

/eEuence alignment


82/162

netzob org@Netzob

e initial ize te matri>

]0 B 2= IG b! BI ]a !2 00

0 0 0 0 0 0 0 0 0 0

]0 0

cH 0

00 0

00 0

/eEuence alignment


83/162

netzob org@Netzob

e =i l l te matri> ((i1: K1, L /:(i: K1, L : (i1: K, L ,

/8 atcMi*matc *core (LM 10,

8 "a *core (0,

/eEuence alignment


84/162

netzob org@Netzob

e =i l l te matri> ((i1: K1, L /: (i: K1, L : (i1: K , L ,

/eEuence alignment


85/162

netzob org@Netzob

e =i l l te matri>

]0 B 2= IG b! BI ]a !2 00

0 0 0 0 0 0 0 0 0 0

]0 0 10 10 10 10 10 10 10 10 10

cH 0 10 10 10 10 10 10 10 10 10

00 0 10 10 10 10 10 10 10 10 20

00 0 10 10 10 10 10 10 10 10 20

/eEuence alignment


86/162

netzob org@Netzob

e !o a traceback

]0 B 2= IG b! BI ]a !2 00

0 0 0 0 0 0 0 0 0 0

]0 0 10 10 10 10 10 10 10 10 10

cH 0 10 10 10 10 10 10 10 10 10

00 0 10 10 10 10 10 10 10 10 20

00 0 10 10 10 10 10 10 10 10 20

/eEuence alignment


87/162

netzob org@Netzob

e comute te common attern

]0 B 2= IG b! BI ]a !2 00

0 0 0 0 0 0 0 0 0 0

]0 0 10 10 10 10 10 10 10 10 10

cH 0 10 10 10 10 10 10 10 10 10

00 0 10 10 10 10 10 10 10 10 20

00 0 10 10 10 10 10 10 10 10 20

]0 B 2= IG b! BI ]a !2 00

]0 cH 00 00

/eEuence alignment


88/162

netzob org@Netzob

e =inall+ buil! a rege>

]0B 2= IG b! BI ]a !2 00

]0cH 00 00

(]0, (.[2:]W, (00,

Static Fields


89/162

netzob org@Netzob

/+mbol > : 0>]0 : (.,[H:1HW:0>00 : ..._

Static Fields


90/162

netzob org@Netzob

()namic Fields

/+mbol > : 0>]0: ( .,[H:1HW : 0>00: ..._


91/162

netzob org@Netzob

%o< to mea*ure *imilarit+ bet


92/162

netzob org@Netzob

ea*ure te @ualit) of Fields

0 U ^ /imilarit+ /core ^ 100 U

e**age* are

i!entical*

e**age* ae

Noting in common


93/162

/imilarit+ *core* bet


94/162

netzob org@Netzob

/18 ratio o= !+namic = iel!* M b+te*

/28 ratio o= common !+namic b+te*

-e !e*ign o= Netzob allo


95/162

netzob org@Netzob

]0B 2= IG b! BI ]a !2 00

]0cH 00 00

/1 \ 1M ( 1L 2,

/ 8 at o o




96/162

netzob org@Netzob

]0B 2= IG b! BI ]a !2 00

]0cH 00 00

/2 \ 2M ]




97/162

netzob org@Netzob


100

100

/1

/2

Normalize!

*imilarit+ *core 8 /


98/162

netzob org@Netzob

%o< to retriee grou* o= *imilar me**age* ?

%ierarcical lu*tering b+ *imilaritie*8

/imilarit+ matri>


99/162

net ob org@Net ob

#6"7

Filling o= a *imilarit+ matri>

$teratiel+ merge te 2 mo*t *imilar me**age*

#6"7 create* a *imilarit+ tree


100/162

t b@N t b


101/162

O7cce** &>amle

e*ult* o= lu*tering an! /eEuence 7lignment


102/162

t b@N t b

e*ult* o= lu*tering an! /eEuence 7lignment


103/162

t b@N t b

A!stract fields

to decontetualie messages


104/162

Full me**age=ormat mo!el


105/162

b@Netzob


106/162

b@Netzob

$nterlu!e

'cauze LM I0 *li!e* le=t

5et*'* !o G minute* o= knitting


107/162

netzob.org@Netzob


108/162


109/162

netzob.org@Netzob

o$ to handle

&nco!e! alue*

(A4: 7/N.1, ?


110/162


111/162

-e i!ea


112/162

netzob.org@Netzob

-ran*=orm ra< b+te* into alicationleel b+te*

7lie! eiter on me**age*: la+er* or =iel!*

6roi!e! =unction* (ba*eIH: gzi: bz2: ,

7llo< cu*tom tran*=ormation =unction*

-e i!ea


113/162

netzob.org@Netzob






114/162

-e i!ea


115/162

netzob.org@Netzob






116/162

7!!ing a cu*tom tran*=ormation =unction

%+ ,ero(ccess -./01ased o1fuscation


117/162

netzob.org@Netzob


118/162

netzob.org@Netzob

/earc =or relation*


119/162

netzob.org@Netzob

%o< to an!le


120/162

netzob.org@Netzob

$nter/+mbol an! $ntra/+mbol relation*

Filename*: etc)


121/162

netzob.org@Netzob

$nter/+mbol an! $ntra/+mbol relation*

/ize Fiel!*: *: etc)


122/162

netzob.org@Netzob

orrelate =iel!'* *ize an! alue* imal $n=ormation oe==icient (.$.N.&.,

Juali=+ correlate! =iel!*

T%e idea


123/162

"enerate 6air* o= !ata =or eac =iel! 8

i l


124/162

netzob.org@Netzob

/imle


125/162

netzob.org@Netzob

Search for closest pairs

ea*ure !een!ence* bet


126/162

netzob.org@Netzob

$N&(Palue(F1, : /ize(F2, , \ 1

T)pical Sie Field Relation

*'.0Value)(*+, ,R,-.)Value)(/+0 Value)(.++7 D 4

T)pical CRC Relation

$N&( , 1


127/162

netzob.org@Netzob

$N&(Palue(F1, : /ize(F2, , \ 1

T)pical Sie Field Relation

*'.0Value)(*+, ,R,-.)Value)(/+0 Value)(.++7 D 4

T)pical CRC Relation


128/162

netzob.org@Netzob

&nironmental !een!encie*


129/162


130/162

netzob.org@Netzob

Step 3 > R0 grammar


131/162

netzob.org@Netzob

/eEuence o= ali! e>cange! *+mbol*.

$4 7utomata

/eEuence o= ali! e>cange! *+mbol*


132/162

netzob.org@Netzob

/eEuence o= ali! e>cange! *+mbol*.$4 7utomata

State 4Attac1 2

Success

State 2

3 t ! ! t i t


133/162

netzob.org@Netzob

3ut an*


134/162

netzob.org@Netzob

4ur mo!el (S**(T, 7!! robabilitie* on outut me**age*

State 4Attac1 2

State 2

34 5 6 (ail.4 5 6 Success

4ur mo!el (S**(T,


135/162

netzob.org@Netzob

4ur mo!el (S**(T, 7!! te reaction time

State 4Attac1 2

State 2

34 5 6 (ail ).444ms+.4 5 6 Success )/4ms+

7ctie "rammatical $n=erence 6roce**

7ngluin 5a 7lgoritm


136/162

netzob.org@Netzob

7ctie "rammatical $n=erence 6roce**

7ngluin 5a 7lgoritm


137/162

netzob.org@Netzob


138/162

netzob.org@Netzob

:enerating traffic"""

Netzob can generate tra==ic tat8


139/162

netzob.org@Netzob

Netzob can generate tra==ic tat8

Follo


140/162

netzob.org@Netzob

3ot client(*, an! *erer


141/162

7b*traction =rom te communication cannel


142/162

netzob.org@Netzob

#/3 cannel

-6 me**age*

a< =ile

$6 =lotual alue* ($6: time: etc.,

7b*traction an! conte>tualization rincile*

$nut !eice

$nut=lotualization rincile*

$nut !eice

$nut

Pocabular+

Pocabular+ lock

lock


146/162

netzob.org@Netzob

ommunication

cannel librar+

ommunication

cannel librar+7b*traction

la+er

7b*traction

la+er"rammar mo!el

(/D-,

"rammar mo!el

(/D-,

$nut

*+mbol*

4utut

*+mbol*

ret5 *+mbol

get5 *+mbol


$nut !eice

$nut

Pocabular+

Pocabular+

lock

lock


147/162

netzob.org@Netzob

ommunication

cannel librar+

ommunication

cannel librar+7b*traction

la+er

7b*traction

la+er"rammar mo!el

(/D-,

"rammar mo!el

(/D-,

$nut

*+mbol*

4utut

*+mbol*

emor+emor+

reiou* eer* $6


$nut !eice

Pocabular+

Pocabular+

lock

lock


148/162

netzob.org@Netzob

ommunication

cannel librar+

ommunication

cannel librar+

4utut !eice

7b*traction

la+er

7b*traction

la+er"rammar mo!el

(/D-,

"rammar mo!el

(/D-,

emor+emor+

4utut

=locange*

Fin! *imilar me**age*

/lit me**age* in =iel!*

7b*tract Fiel!*

/earc =or relation*

(0*ES

Demo 28 retriee te 626 zombie !irector+

/imulation o= a reali*tic zombie


151/162

netzob.org@Netzob

/imulation o= a reali*tic zombie

a te eer* neigbour* relation*


152/162

$ntegrate! *mart =uzzing: b+ leeraging te

*imulator engine


153/162

netzob.org@Netzob

7llo


154/162

netzob.org@Netzob

7llo


155/162

netzob.org@Netzob

#/3

$4-5

76$ (**l)rea!: **l)ort rotocol mo!el in more r! art+

ro!uct* 4coming soon5


156/162

netzob.org@Netzob

ire*ark

/ca+

6eac Fuzzer

7llo


157/162

netzob.org@Netzob

ire*ark

/ca+

6eac Fuzzer

7llo


158/162

netzob.org@Netzob

Conclusion"""

6rotocol & automation !omain i* Euite actie

at te aca!emic leel


159/162

netzob.org@Netzob

3ut no real tool aailable ...

Netzob trie* to =ill ti* lack b+

/uorting aca!emic re*earce*

3eing u*able in oerational conte>t

4en to all kin! o= contribution*

Fee!back


160/162

netzob.org@Netzob

3ug =i>

Feature roo*al M imlementation

-ran*lation

...

Than1s for )ou attention &Than1s for )ou attention &

An) =uestions -An) =uestions -


161/162

netzob.org@Netzob

An) =uestions -An) =uestions -

$$$"neto!"org$$$"neto!"org?neto!?neto!

$mage licence*tt8MM


162/162

netzob.org@Netzob

3YN r tK.black

Date post:	02-Jun-2018
Category:	Documents
Upload:	noscribdyoucant
View:	217 times
Download:	0 times

Netzob Tutorial

Documents