+ All Categories
Home > Documents > Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save...

Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save...

Date post: 21-Apr-2020
Category:
Upload: others
View: 18 times
Download: 0 times
Share this document with a friend
22
Erasure Code in Ceph Loic Dachary @ Red Hat
Transcript
Page 1: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Erasure Code in CephLoic Dachary @ Red Hat

Page 2: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Save Space

Page 3: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

5 minutes role playing game

Page 4: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

XOR

Page 5: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

3 peta => 1.3 peta

Page 6: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Harder object mutations / recovery

Page 7: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Simple operations and tiering

Replicated Erasure Coded

Page 8: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Promoted to replica on read

Replicated Erasure Coded

Page 9: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Sam & David : internals

Page 10: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Janne & Andreas & Loic & Takeshierasure code

Page 11: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Released May 2014 : Firefly

Page 12: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Why save space before shortage ?

Page 13: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Reliability Model

● After an OSD is lost● Recovery● Backfilling a new OSD

Page 14: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

April 2015 : Hammer

Page 15: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Repair K=10, M=4

Page 16: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Locally Recoverable CodesLRC @ Red Hat

Page 17: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

SHEC Takeshi @ Fujitsu

Page 18: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

ISA plugin Yuan @ Intel

SIMD aka SSE2, SSE3, SSE4

Only for Intel processors

~50% Faster

Page 19: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

jerasure ARM Janne @ ARM

Page 20: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Infernalis

Page 21: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

Hitchhiker Rashmi @ U.C. Berkeley

Page 22: Erasure Code in Ceph Loic Dachary @ Red Hat · Erasure Code in Ceph Loic Dachary @ Red Hat. Save Space. 5 minutes role playing game. XOR. 3 peta => 1.3 peta. Harder object mutations

[email protected]

Artwork GPLv3+ Tartaruga Feliz


Recommended