Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | tylor-colleton |
View: | 218 times |
Download: | 0 times |
On-the-fly Verification of Erasure-Encoded
File Transfers
Mike Freedman & Max KrohnNYU Dept of Computer Science
Downloading Large Files From P2P Networks For large files, transfer times are much
bigger than average node uptimes.
Some files are very popular: multiple sources and multiple requesting nodes.
Is it possible to have multicast, even though sources and receivers frequently enter and leave the network.
Solution: Rateless Erasure Codes
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Solution: Rateless Erasure Codes
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
( )h F
( )h F ( )h F( )h F
Wants file F
Mutli-Sourced Downloads
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Mutli-Sourced Downloads
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Receiver (R3) Receiver (R4)
Receiver (R3)
“Overlapping Multicast Trees”
Source (S1) Source (S2) Source (S3) Source (S4)
Receiver (R2)
Receiver (R1)
Resuming Truncated Downloads
Source (S1)
Receiver (R1) Receiver (R2)
Resuming Truncated Downloads
Source (S1)
Receiver (R1) Receiver (R2)
Resuming Truncated Downloads
Source (S1)
Receiver (R1) Receiver (R2)
Threat Model
KaZaaKaZaa
eDonkey 2000
Gnutella
Morpheus
Threat Model
KaZaaKaZaaeDonkey 2000
Gnutella
Morpheus
Threat Model
KaZaaKaZaaeDonkey 2000
Gnutella
Morpheus
Threat Model
KaZaaKaZaa
eDonkey 2000
Gnutella
Morpheus
Bogus Data Attack
KaZaaKaZaaeDonkey 2000
Gnutella
Morpheus
Unwanted Data Attack
KaZaaKaZaaeDonkey 2000
Gnutella
Morpheus
Attacking Erasure Encoded Transfers
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Attacking Erasure Encoded Transfers
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Erasure Encoding of Files
1b 2b 3b 4b
1c 2c 3c 4c 5c 6c
1 2 3
2 2 3 4
c b b
c b b b
3 1 4
4 3 4
c b b
c b b
F
…
Easily Verifiable….
1( ) ( ( ),..., ( ))nh F h b h b
1b 2b 3b 4b
1c 2c 3c 4c 5c 6c
F
…
…but Not on the Fly
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
What Happened?R1 received checkblock c from
S4. S4 claims:
blah
9813024 bbbc
What Happened?R1 received checkblock c from
S4. S4 claims:R1 knows:
But how can R1 verify c?
Wouldn’t it be nice if:
Not true for SHA1!
9813024 bbbc )(),(),( 9813024 bhbhbh
What Happened?R1 received checkblock c from
S4. S4 claims:R1 knows: But how can R1 verify c?
Wouldn’t it be nice if:
Not true for SHA1!
9813024 bbbc )(),(),( 9813024 bhbhbh
What Happened?R1 received checkblock c from
S4. S4 claims:R1 knows: But how can R1 verify c?Wouldn’t it be nice if:
Not true for SHA1!
9813024 bbbc )(),(),( 9813024 bhbhbh
)()()()( 9813024 bhbhbhch
What Happened?R1 received checkblock c from
S4. S4 claims:R1 knows: But how can R1 verify c?Wouldn’t it be nice if:
Not true for SHA1!
9813024 bbbc )(),(),( 9813024 bhbhbh
)()()()( 9813024 bhbhbhch
A Homomorphic Hashing Scheme Assume file block size of 8kB
Pick large prime (about 1024 bits) and small prime (about 256 bits) that divides , and 256 generators of order q:
Writes the file F as matrix, elements in
pq
1,1 1,
1
256,1 256,
( )n
n
n
b b
b b
F b b
( 1)p
1 256( ,..., )g gg
qZ
How To Hash The hash of a message or check block is
an element in :qZ
1,1
2,1
1,1 2,1 256,1
256,1
1
11,1
2,1 21 2 256
256,1256
( )
( )( ) ( )x
b
bb b b
b
h
gb
b gg g g
b g
g
b
Π
How To Hash The hash of a message or check block is
an element in :
The hash of the entire file is an n-element
vector of the hashes of the blocks:
1( ) ( ( ),..., ( ))nH h h h F b b
,
256
1
( ) (mod )i kbk i
i
h g p
b
qZ
The Only Important Slide
( ) ( ) ( )j k j kh h h b b b b
,
256
1
( ) (mod )i kbk i
i
h g p
b
implies that
Why?
a b a bg g g
How To Encode Checkblocks are constructed using
modular addition over .
To generate a checkblock, pick a set
{1,..., }S n
(mod )S kk S
q
c b
qZ
And compute
How To VerifyGiven the correct hash:
And a check block:
verify that:
Note: LHS computation is expensive!
1( ( ),..., ( ))nH h h b b
( ) ( )S kk S
h h
c b
(mod )S kk S
q
c b
Success!
Source (S1)
Receiver (R1)
Source (S2) Source (S3) Source (S4)
Analysis+ Security of the hash function based on
hardness of the discrete log.
− Hashes are big (1/256 the size of the file), but we can apply this process recursively.
+ Our paper details a batched, probabilistic verification scheme that drastically reduces exponentiations.
+ Verifying rate is 40x faster than download rates on a T1.