22
RAID-Z Expansion Matt Ahrens
RAID-Z ExpansionMatt Ahrens
Problem:You have a RAID-Z pool with 4 disks in it
You want to add a 5th disk
How does traditional RAID 4/5/6 do it?
1 2 3 4 P1-4
5 6 7 8 P5-8
9 10 11 12 P9-12
13 14 15 16 P13-1
6
17 18 19 20 P17-2
0
1 2 3 P1-3
4 5 6 P4-6
7 8 9 P7-9
10 11 12 P10-1
2
13 14 15 P13-1
5
Color indicates parity group (stripe)
RAID-Z Expansion: Reflow
1p 2 3 4 5p
6 7p 8 9 10
11p 12 13p 14 15
16 17p 18 19 20
21 22 23 24 25
1p 2 3 4
5p 6 7p 8
9 10 11p 12
13p 14 15 16
17p 18 19 20
Color indicates parity group (logical stripe)
RAID-Z Expansion: Reflow copies allocated data
1p 2 3 4 5p
6 7p 8 9 10
11 12 13 14 15
16 17p 18 19 20
21 22 23 24 25
1p 2 3 4
5p 6 7p 8
9 10 11 12
13 14 15 16
17p 18 19 20
Color indicates parity group (logical stripe)
This works!● Doesn’t change block pointers● Reads / Writes sequentially● Spacemaps tell us what we need to copy● Each logical stripe is independent
○ Don’t need to know where parity is○ Segments still on different disks, so redundancy is preserved○ (contraction couldn’t work this way)
RAID-Z Expansion: new writes, new stripe width
1p 2 3 4 5p
6 7p 8 9 10
11p 12 13 14 15
16x 17p 18 19 20
21 22 23 24 25
1p 2 3 4
5p 6 7p 8
9 10 11 12
13 14 15 16
17p 18 19 20
Color indicates parity group (logical stripe)
Logical vs Physical stripe width● After conversion, physical stripe width is 5● Old blocks still have logical stripe width of 4● New blocks have logical stripe width of 5
○ Improved data : parity ratio (4:1 instead of 3:1)
● When reading, need to know logical stripe width○ Use block’s birth time (+ expansion time) to determine
Reflow Progress = 4; Chunk Size = 1
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 5; Chunk Size = 1
1 2 3 4 5
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 6; Chunk Size = 1
1 2 3 4 5
6 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 7; Chunk Size = 1
1 2 3 4 5
6 7 7 8
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 8; Chunk Size = 2
1 2 3 4 5
6 7 8 8
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 10; Chunk Size = 2
1 2 3 4 5
6 7 8 9 10
9 10 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 12; Chunk Size = 3
1 2 3 4 5
6 7 8 9 10
11 12 11 12
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 15; Chunk Size = 3
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
13 14 15 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 18; Chunk Size = 4
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 16
17 18 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow Progress = 22; Chunk Size = 5
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 19 20
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
Color indicates logical stripe; white=unused
Reflow process● Need to track progress to know the physical stripe width● Each TXG can only overwrite what’s previously unused● Exponential increase in amount that can be copied per TXG
○ 41 TXG’s for 1MB○ 113 TXG’s for 1GB○ 186 TXG’s for 1TB○ 258 TXG’s for 1PB
Design implications● Works with RAIDZ-1/2/3● Can expand multiple times (4-wide -> 5 wide -> 6 wide)● Old data has old Data : Parity ratio● New data has new Data : Parity ratio● RAIDZ must be healthy (no missing devices) during reflow
○ If disk dies, reflow will pause and wait for it to be reconstructed
● Reflow works in the background
Thank you!
Status● High level design complete● Detailed design 80% complete● Zero lines of code written● Expect significant updates at BSDCAN (June 2018) and next year’s
DevSummit (~October 2018)
![P v Ç Z µ ] v P d Z W } i W Z ] o ^ ( Ç u v PIJ... · 2020-04-10 · z z z E z z z z E z z z {W } P u D v P u v ~ } v } }W } P u D v P u v ~ v µ Z o o } i u ] v ] v } v ] v }](https://static.documents.pub/doc/80x56/5f29c6618e2932551e389057/p-v-z-v-p-d-z-w-i-w-z-o-u-v-pij-2020-04-10-z-z-z-e-z-z.jpg)
![Wd E D ^z>> h^ Z u ] Ç ] o Ç d í ì ( } z o z í î Z z v P P ... · Wd E D ^z>> h^ Z u ] Ç ] o Ç d í ì ( } z o z í î Z z v P P ... ... 3 o,](https://static.documents.pub/doc/80x56/60358b60a9dc4417a41aab0e/wd-e-d-z-h-z-u-o-d-z-o-z-z-z-v-p-p-wd-e.jpg)
![The Translators to the Reader · K v h v ] ( } u ] Ç } ( W Z ] v P v } Z Z ] v P Á Z ] v l P } } } u } v ] Z Z } ( ~ P v o Z Z Á K v h v ] ( } u ] Ç } ( W Z ] v P r Æ u o ·](https://static.documents.pub/doc/80x56/5f14ea764296f15d5e78f92e/the-translators-to-the-reader-k-v-h-v-u-w-z-v-p-v-z-z-v-p-.jpg)
![Ã 1 p · > v P l Z r o v P l Z u v P Z ] h i ] v E ] } v oH > v P l Z r o v P l Z u v P Z ] h i ] v E ] } v oH](https://static.documents.pub/doc/80x56/5f8d948c7f1c412edc026d69/f-1-p-v-p-l-z-r-o-v-p-l-z-u-v-p-z-h-i-v-e-v-oh-v-p-l-z-r-o-v.jpg)
![Wd E D ^z>> h^ Z u ] Ç ] o Ç d í ì ( } z o z í î Z z v P P ......Wd E D ^z>> h^ Z u ] Ç ] o Ç d í ì ( } z o z í î Z z v P P ... ... 3 o,](https://static.documents.pub/doc/80x56/60358cc77dbdfe67ff50c666/wd-e-d-z-h-z-u-o-d-z-o-z-z-z-v-p-p-wd.jpg)
