1
REFERENCES [1] MY Ahmad and B Kemme. Compaction Management in Distributed Key-Value Datastores. PVLDB 8(8):850–861, 2015. [2] AS Aiyer, M Bautin, GJ Chen, P Damania, P Khemani, K Muthukkaruppan, K Ranganathan, N Spiegelberg, L Tang, and M Vaidya. Storage infrastructure behind Facebook Messages: Using HBase at scale. IEEE Data Eng. Bull., 35(2):4–13, 2012. [3] F Chang, J Dean, S Ghemawat, WC Hsieh, DA Wallach, M Burrows, T Chandra, A Fikes, and R Gruber. Bigtable: A distributed storage system for structured data. In OSDI, pages 205–218, 2006. [4] PE O'Neil, E Cheng, D Gawlick, and EJ O'Neil. The log-structured merge-tree (LSM-tree). Acta Inf., 33(4):351–385, 1996. [5] Apache HBase. http://hbase.apache.org/ ... RESULTS – Compaction server assumes execution and overheads. – Compactions are shorter; read latency improves. – Cache misses less costly; reads faster over network vs. disk. – Incremental warmup eliminates cache misses altogether. – Multiple compaction servers allow for load balancing. ACKNOWLEDGEMENTS This work is partially funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) and Ministère de l’Enseignement supérieur, Recherche, Science et Technologie, Québec, Canada. EXPERIMENTS Fig. 2 – Remote Caching Fig. 3 – Incremental Warmup Fig. 4 – Standard 5x RS - No CS Fig. 5 – Offloaded 5x RS - 1x CS Fig. 1 – Standard Compaction SCALABILITY 0 50 100 150 200 0 3600 7200 10800 ) s m ( y c n e t a L Time (s) Get Scan 0 50 100 150 200 0 3600 7200 10800 ) s m ( y c n e t a L Time (s) Get Scan 0 50 100 150 200 0 3600 7200 10800 ) s m ( y c n e t a L Time (s) Get Scan 0 50 100 150 200 0 3600 7200 10800 ) s m ( y c n e t a L Time (s) Get Scan 0 50 100 150 200 250 0 300 600 900 ) s m ( y c n e t a L Time (s) Get Scan 0 50 100 150 200 250 0 300 600 900 ) s m ( y c n e t a L Time (s) Get Scan Compactions pile up, overloading the single compaction server. – YCSB update workload triggers compactions. – YCSB read worklaod measures get/scan latency. – 1x region server (RS), 1x compaction server (CS). Fig. 6 – Under-Provisioned 10x RS - 1x CS Fig. 7 – Balanced 10x RS - 2x CS Compactions are distributed across two compaction servers. SOLUTION ≡ Compaction Offloading – Offload compactions to specialized compaction servers. – Dedicate region server resources to workload execution. ≡ Remote Caching – Compaction server caches compaction results locally. – Region server reads back results over network. ≡ Incremental Warmup – Do not evict invalidated file blocks en masse. – Gradually phase old data out, block by block. – Replace with new data from remote cache. – Sequential transfer (files are already sorted). Application HBase Client HBase Master HDFS Master Zoo Keeper Datanode Region Server WAL Block Cache Mem Datanode . . . . . . Region Server WAL Mem Block Cache Compaction Manager Compaction Server Datanode Block Cache COMPACTIONS 0 50 100 150 200 250 0 300 600 900 ) s m ( y c n e t a L Time (s) Get Scan 1. Reduce compaction overheads on region server. 2. Prevent large spikes in read latency. GOALS Periodic data maintenance for multi-version stores – Consolidate updates into existing dataset Prevent read latency from degrading over time – Reduce number of overlapping data files to be read Compactions are expensive! 1. During execution: compete with workload for resources 2. After execution: degrade read latency severely – Input files removed, evicted en masse from cache ► Cache misses COMPACT RAM UPDATES COMPACT WARMUP Compacon management in distributed key-value datastores Muhammad Yousuf Ahmad Bena Kemme [email protected] [email protected]
![Page 1: Compaction management in distributed key-value datastoresacmsocc.github.io/2015/posters/socc15posters-final34.pdf · 2019-03-18 · REFERENCES [1] MY Ahmad and B Kemme. Compaction](https://reader034.documents.pub/reader034/viewer/2022042309/5ed6e4dbdf0eda5e752aea5e/html5/thumbnails/1.jpg)
