Date post: | 11-Jan-2017 |
Category: |
Software |
Upload: | alluxio-inc |
View: | 345 times |
Download: | 4 times |
The Missing Piece of On-Demand Clusters
October 2016
Calvin Jia
About Me
• Calvin Jia
• Software Engineer @ Alluxio, Inc.
• Alluxio PMC
• #1 Alluxio Contributor
• Twitter: @JiaCalvin
2
Outline
• Cloud Architectures• Alluxio in Cloud Architectures• Takeaways
3
Cloud Architectures – Overview
• Mostly service based, from providers– Amazon Web Services– Google Cloud Platform
• Separate compute and storage clusters• Compute clusters are ephemeral
4
Cloud Architectures – Pros & Cons
Pros• Low maintenance • Pay as you go• Elastic and scalable• Cost effective storage
Cons• Lower Performance
5
Outline
• Cloud Architectures• Alluxio in Cloud Architectures• Takeaways
6
Alluxio in Cloud Architectures – Overview
• Deployed in compute clusters– Memory speed data access– Transparent data access to any storage
• Simple to deploy– Mount storage systems similar to local disks
7
Alluxio in Cloud Architectures – Example
8
Alluxio in Cloud Architectures – Benefits
9
• Remedies the performance drawback• Acceleration due to memory-speed I/O• Designed to improve the affinity of compute
and storage
Alluxio in Cloud Architectures – Data Path
10
FAST 104 - 105 MB/s
MODERATE 103 - 104 MB/s
SLOW 102 - 103 MB/s
Only when necessaryLimited
Often
SSDHDD
Mem
Takeaways – Experiment Results
11
0
100
200
300
400
500
600
Spark - No PersistSpark - PersistAlluxio
Runti
me
(sec
onds
)
Takeaways – Alluxio & Cloud Architectures
12
• Cloud architectures have significant upsides• Alluxio alleviates the major downsides
Try out Alluxio 1.2.0http://www.alluxio.org/releases
13
14
Read the blog at http://www.alluxio.com/2016/08/accelerating-on-demand-data-analytics-with-alluxio
Read the whitepaper at www.alluxio.com.
Check out www.alluxio.org.
Thank you!