Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 213 times |
Download: | 0 times |
Agenda
Background
Redundant Data Regeneration
Sequential Redundant Data Update(SRDU)
Update Overhead Analysis
Performance Evaluation
Conclusion and Future Works
BackgroundData Placement
v0 v1 v2 v3 c0,0 c0,1
d2d1d0 d3 r0 r1
v4
v8
v12
c1,0
c2,0
c3,0
c1,1
c2,1
c3,1
v5 v6 v7
v9 v10 v11
v13 v14 v15
v16 c4,0 c4,1v17 v18 v19
Same Parity Group
Data Nodes Redundancy Nodes
Q-bytes data blocks Q-bytes
redundant data blocks
BackgroundNew nodes join the system
Data blocks are reorganized to utilize storage and streaming capacity
Data blocks in Parity Group change
Redundant data blocks need to update/re-compute
v0 v1 v2 v3 c’0,0 c’0,1
d2d1d0 d3 r0 r1
v5
v10
v15
c’1,0
c’2,0
c’3,0
c’1,1
c’2,1
c’3,1
v6 v7 v8
v11 v12 v13
v16 v17 v18
v4
d4
v9
v14
v19
Data blocksin parity group CHANGE!
Redundant data blocks CHANGE!
Redundant Data Regeneration
Regenerate new redundant blocks by all the data blocksData blocks are distributed among nodes
Send all the data blocks to redundancy nodeCompute the new redundant
blocks in redundancy nodeOverhead is too high
Redundancy Node
Data Nodes
...
Redundant Data RegenerationData blocks are stored in central archive
serverRegenerate all the new redundant data in serverSend to redundancy nodesServer may be costly, not desirable, not feasible
Redundancy Node
Server
Sequential Redundant Data UpdateIdeaOld redundant data blocks contain valuable
information on the data blocksData blocks are different after reorganizationBut some blocks in parity group is the SAMEPossible to reuse for linear code
v0 v1 v2 v3 c’0,0 c’0,1v4
New Redundant data blocks
v0 v1 v2 v3 c0,0 c0,1
Old Redundant data blocks
d2d1d0 d3 r0 r1d4
d2d1d0 d3 r0 r1
Sequential Redundant Data UpdateLinear erasure correction code Can be analyzed using linear algebra Linear matrix multiplications in computing redundant data
Reed-Solomon Erasure Correction Code Redundant data by Vandermonde matrix
Number of nodes, N Number of redundancy node, h Element at row i, column j = ji-1
Data from data node 0
Redundant data in redundancy node 0
,0 ,0
,1 ,1
1 1 1, 1 , 1
1 1 1 1
1 2 3
1 2 3 ( )
i i
i i
h h hi h i N h
c d
c dN h
c dN h
Rounding or truncating would prevent a correct reconstruction of source data 1 / 3 = 0.3333, decoding fails
Arithmetic over Galois Fields Finite Fields
Finite elements Add, subtract, multiply, divide similar to algebra Closed under these operations
GF(2w) are integers from zero to 2w-1 Fixed bits to represent an element
Assume a Q-bytes data block can be represented by a field element
Sequential Redundant Data Update
Sequential Redundant Data Update(SRDU) consists of 3 techniquesDirect reuse of old redundant dataParity group reshufflingCaching of transmitted data
Sequential Redundant Data Update
SRDU – Direct Reuse
v0 v1 v2 v3 c’0,0 c’0,1v4
v0 v1 v2 v3 c0,0 c0,1
d2d1d0 d3 r0 r1d4
d2d1d0 d3 r0 r1
0
0,0 1
0,1 2
3
1 1 1 1
1 2 3 4
v
c v
c v
v
0
10,0
20,1
3
4
' 1 1 1 1 1
' 1 2 3 4 5
v
vc
vc
v
v
0,1 0 1 2 31* 2* 3* 4*c v v v v
Only need to send v4,
Save 4 blocks
0,1 0 1 2 3 4
0,1 4
' (1* 2* 3* 4* ) 5*
5*
c v v v v v
c v
Construct 1st parity group Before node addition
After node addition
SRDU – Direct ReuseConstruct 2nd parity group Before node addition
v5 v6 v7 v8 c’1,0 c’1,1v9
v4 v5 v6 v7 c1,0 c1,1
d2d1d0 d3 r0 r1d4
d2d1d0 d3 r0 r1
4
1,0 5
1,1 6
7
1 1 1 1
1 2 3 4
v
c v
c v
v
5
61,0
71,1
8
9
' 1 1 1 1 1
' 1 2 3 4 5
v
vc
vc
v
v
1,1 4 5 6 71* 2* 3* 4*c v v v v
Terms are different.Direct Reuse Fails!
1,1 5 6 7 8 9' 1* 2* 3* 4* 5*c v v v v v
After node addition
SRDU – Parity Group ReshufflingThe order during encoding is not important
Parity group reshuffling Before node addition
After node addition
5 8
6 5
7 6
8 7
9 9
reshuffle to
v v
v v
v v
v v
v v
1,1 4 5 6 71* 2* 3* 4*c v v v v
1,1 8 5 6 7 9
8 4 5 6 7 4 9
8 1,1 4 9
' 1* 2* 3* 4* 5*
1* (1* 2* 3* 4* 1* ) 5*
1* ( 1* ) 5*
c v v v v v
v v v v v v v
v c v v
Only need to send v4, v8 and v9
Save 2 blocks
SRDU – Parity Group ReshufflingChoices of old redundant data for reusing
5 8
6 9
7 5
8 6
9 7
reshuffle to
v v
v v
v v
v v
v v
Before node addition
After node addition
v5 v6 v7 v8 c’1,0 c’1,1v9
v8 v9 v10 v11 c2,0 c2,1
d2d1d0 d3 r0 r1d4
d2d1d0 d3 r0 r1
2,1 8 9 10 111* 2* 3* 4*c v v v v
1,1 8 9 5 6 7
2,1 10 11 5 6 7
' (1* 2* ) 3* 4* 5*
( 3* 4* ) 3* 4* 5*
c v v v v v
c v v v v v
5 blocks needed,
More overhead
SRDU - CachingCaching of transmitted data in constructing previous redundant block
Reuse them to construct current redundant blocksConstruct 1st parity group
Construct 2nd parity group
0,1 0,1 4' 5*c c v
1,1 8 1,1 4 9' 1* ( 1* ) 5*c v c v v
v4 is cached in the redundancy node
v4 can be reused from locally cached data,Only v8 and v9 need to send
SRDU - CachingThe buffer for caching is at most (N-h)
Data blocks from earlier than (N-h) blocks are not useful for reusingConstruct the 4th parity group
15 16 17 18 19( , , , , )v v v v v
16 17 18 19( , , , )v v v v12 13 14 15( , , , )v v v v
c4,1c3,1
All previous blocks are useless
c’3,1
At most 4 data blocks are cached
Old redundant block
New redundant block
SRDU - Summary
For construction of each new redundant blockReuse old redundant block by
Direct reuseParity group reshufflingCaching of transmitted data
Select the one with lowest overhead to reuse
Send the required blocks only
Update Overhead AnalysisTotal data blocks, BRedundant Data Regeneration Data blocks are distributed among nodes
Total overhead = B + (h-1) * B/(N-h)
All data blocks are sent, overhead = B
All redundant data are constructed here overhead = B / (N-h)
.... . .
...
Update Overhead AnalysisData blocks are stored in central archive
serverTotal overhead = h * B/(N-h)
overhead = B / (N-h)
All redundant data are constructed in server
......
Update Overhead AnalysisSequential Redundant Data Update(SRDU)Total overhead = block movement under SRDU
+ (h -1) * B/(N-h)
Data blocks are sent under decision of SRDU
Compute partial result of each new redundant blocks
overhead = B / (N-h)
Combine the old redundant data and received partial result
1,1 8 1,1 4 9
1,1
' 1* ( 1* ) 5*
c v c v v
c P
8 4 91* 1* 5*P v v v
.... . .
...
Update Overhead AnalysisRedundant data RegenerationData blocks are distributed among nodes
Total overhead = B + (h-1) * B/(N-h)Data blocks are stored in central archive server
Total overhead = B/(N-h) + (h-1) * B/(N-h)
Sequential Redundant Data Update(SRDU)Total overhead = blocks movement under SRDU +
(h-1) * B/(N-h)
Sufficient to consider overhead for one redundant update as performance metric
Performance EvaluationUpdate overhead in continuous system growth
Number of data blocks = 40000Grow from 1 node to 400 nodes, one node added each
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
0 100 200 300 400
System Size (nodes)
Red
un
dan
cy
Up
date
Ov
erh
ead
(b
lock
s)
Regeneration by Fully Distributed Data
Regeneration by a Archive Server
Direct reuse + Reshuffling
Direct Reuse + Reshuffling + Caching
Direct Reuse
Performance EvaluationUpdate overhead in batched redundancy update
Defer update until a fixed number of nodes, W Initial system size = 80, W ranges from 1 to 200
100
1000
10000
100000
0 50 100 150 200
batch size W (nodes)
Per
-no
de
Red
un
dan
cy U
pd
ate
Ov
ehea
d (
blo
cks)
Direct Reuse + Reshuffling + Caching, Starting at 80
Direct Reuse + Reshuffling,Starting at 80 Direct Reuse,
Starting at 80
Conclusion
Identify the redundant data update problem
Investigate redundant data regeneration
Propose SRDU to reduce the overhead substantially (about 75%)
Further reduce the overhead by batched redundancy update (about 97% for a batch size of 10)