November 12, 2014 | Las Vegas, NV
Guy Farber, AWS Storage Business Development
AWS storage options
•
•
•
•
• Customer presentation: Ooyala
We are constantly producing more data
From all types of industries
AMAZON S3Simple Storage Service
Of Unique Customer Objects
Q4 2006
Q1 2007
Q2 2007
Q3 2007
Q4 2007
Q1 2008
Q2 2008
Q3 2008
Q4 2008
Q1 2009
Q2 2009
Q3 2009
Q4 2009
Q1 2010
Q2 2010
Q3 2010
Q4 2010
Q1 2011
Q2 2011
Q3 2011
Q4 2011
Q1 2012
Q2 2012
Q3 2012
Q4 2012
Q1 2013
Q2 2013
Q3 2013
peak transactions per second
durability
99.99% saves ~20%
Amazon S3
Amazon S3
1 PB raw storage
800 TB usable storage
600 TB allocated storage
400 TB application data
S3 pay only for what you use!
Amazon S3
Amazon S3
Amazon S3
Amazon S3
Amazon S3
Amazon
CloudFront
Low-cost archiving service
per GB / month
per TB / year
durability
data retrieval
vs.
Amazon Glacier
Policy-based archiving service
Amazon S3 →
Amazon Glacier
• World’s leading social sound
platform
• Audio files must be transcoded and
stored in multiple formats
S3
Amazon Glacier
S3
•
•
• Amazon Glacier
Elastic Block Storage
10GB1TB
Price Performance
Latency Business impact
EBS
Magnetic General Purpose Provisioned IOPS
Use Cases Infrequent data access
Boot volumes
Small to Med DBs
Dev and Test
I/O intensive
Relational DBs
NoSQL DBs
Storage Media Magnetic disk-backed SSD-backed SSD-backed
Performance 100 IOPS on average.
Ability to burst to
hundreds of IOPS
Burst to 3,000 IOPS
maximum, with a base
performance of 3 IOPS/GiB
Consistently performs at
provisioned level, 4000
IOPS maximum
Price$.05/GB - Month
$.05/million I/O$.10/GB – Month
$.125/GB - Month
$.065/provisioned IOPS
Amazon EBS
Amazon EC2
AWS Storage Gateway
Your on-ramp to AWS cloud storage:• Backup into Amazon S3
• DR on Amazon EC2
• Archive into Amazon Glacier
For a deeper dive, join: BAC202 - Introducing AWS
Solutions for Backup and Archiving
Performance-orientedCost-oriented
CLOUD STORAGE
AT OOYALA
Ilan Rabinovitch
Manager, Infrastructure and Site Reliability Engineering
HISTORY OF AWS AT OOYALA
• Ooyala born in the cloud in 2007
• Winner of the first AWS Startup Challenge in 2007
• Our mission is to help our customers bring
personalized video to every screen everywhere
• Today we use a number of AWS services
including:
• Amazon S3
• Amazon EC2 + Amazon VPC
• Amazon CloudFront
• AWS Direct Connect
Storage use cases
Log storage with Amazon S3 and Amazon Glacier
• As a data-focused company, we log and analyze
EVERYTHING
• Needed a scalable and cost-effective way to transport,
store, and access billions of log files
• Enter Amazon S3 and Amazon Glacier
• Store application logs and telemetry data in S3
• Download and process in Hadoop and Spark
• Archive older data to Glacier to reduce storage
costs
Log storage with Amazon S3 and Amazon Glacier
Infinitely scalable storage with S3
Automatically archive older data
to Glacier via life cycle polices
Low latency connectivity to on-
premises facilities via Direct
Connect
Benefits Tips
Know your access patterns:
Large batch restorations can be
time intensive and costly
Plan ahead! S3 Import / Export
requires you to thaw from Glacier
first
Secure your data with custom
IAM keys and instance roles
WALLE: Safe deletion in S3
• Ooyala stores and transcodes customer videos in S3 as origin for CDNs
• Overtime needed to delete over 200 million stale files safely
• Enter WALLE:
• Internal API for asset deletion
• Spawns Spot Instances based on workload to issue deletion requests
• S3 copy to temporary “Trash” bucket with lifecycle policies
WALLEWALLE: Safe deletion in S3
Tips
• Spring Cleaning Can Generate Real Savings
Automate your data retention and deletion with lifecycle policies
• Safety First
Use S3 versioning or temporary storage to protect against mistakenly
overwriting or deleting data
• Scale Cost-Effectively
Use Spot Instances and Auto Scaling where possible to keep costs in
check
Client-side web applications
• Ooyala develops web applications, which our clients use to
manage their content libraries
• Many applications have migrated to client-side JavaScript to
allow for a richer and more performant user experience
• Needed a simple way to deploy and scale these client-side
applications
• Enter S3 websites and CloudFront
• Upload static content such as HTML + JS to S3
• Accelerate and cache content via Amazon CloudFront
• Use AJAX to interact with server side APIs and data
Client-side web applications
Easy and repeatable deployments
Scales easily and securely; no web
servers to manage
Low-cost, highly performant delivery via
CloudFront
Benefits Tips
Reuse S3 buckets: AWS allows up to 100
S3 buckets per account; use CloudFront
behaviors and IAM to host multiple domains
on a single S3 bucket
Cache-control headers Don’t forget to set
CacheControl headers on your objects to
ensure caching benefits
Secure delivery: Use CloudFront to provide
SSL delivery with custom certificates on
your domain
Learn more about AWS at Ooyala: engineering.ooyala.com
Join our team and build on the cloud: ooyala.com/careers
THANK YOU