Home >Technology >AWS Webcast - Introduction to EBS

AWS Webcast - Introduction to EBS

Date post:15-Jan-2015
View:816 times
Download:0 times
Share this document with a friend
Elastic Block Store is one of the fundamental components of a best-practices cloud architecture. Join Senior Solutions Architect Miles Ward for a detailed review of the service, a clear breakdown on cost and approaches for price estimates, easy methods for extracting maximum performance from both Standard and Provisioned-IOPS EBS volumes, configuration nuances for both Windows and Linux users, RAID guidance, RDBMS and NoSQL storage configuration tuning, as well as several examples of EBS delivering amazing value at scale.
  • 1. Presenter Name Presenter Title Month Day, Year Understanding Amazon EBS Availability and Performance Miles Ward Senior Manager, Solutions Architecture

2. Agenda Overview of Elastic Block Store Some key concepts: block size, IOPS, throughput Performance Availability 3. Storage Options on AWS Block Storage (Elastic Block Store) Object Storage (S3, Glacier) Use for: Access to raw unformatted block level storage Persistent Storage Use for: Pictures, videos, highly durable media storage Cold storage for long-term archive 4. Amazon Elastic Block Store (EBS) Elastic Block Storage: Persistent Storage for EC2 Feature Details High performance file system Mount EBS as drives and format as required Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an Availability Zone Backups Volumes can be snapshotted for point in time restore Monitoring Detailed metrics captured via Cloud Watch 5. What are some of our customers doing with EBS? Enterprises Enterprise workloads are built on block storage Oracle, SAP, Microsoft Applications Convenient, cost-effective, reliable file server Gaming/Social/ Mobile/Education Very high performance and consistent IO for NoSQL and relational DBs Marketing / Analytics Fast sequential IO access 6. Key EBS concepts Standard EBS Volumes Provisioned IOPS EBS Volumes Block Size Queue Depth Snapshots 7. Standard and Provisioned IOPS Volume Types Standard Volumes Provisioned IOPS Volumes Optimized for Workloads with low or moderate IOPS needs and occasional bursts. Transactional workloads requiring consistent IOPS. Volume Attributes Up to 1 TB, average 100 IOPS per volume. Best effort performance. Can be striped together for larger size and higher IOPS. Up to 1TB, 2,000 IOPS per volume. Consistent IOPS. Can be striped together for larger size and higher IOPS. Workloads File server, Log processing, Websites, Analytics, Boot, etc. Business applications, MongoDB, SQL server, MySQL, Postgres, Oracle, etc. 8. Introducing Provisioned IOPS Volumes Select a new type of Provisioned IOPS volumes Specify the number of IOs per second your application needs, up to 4000 PIOPS per volume. The volume will deliver the specified IO per second. Specify the volume capacity $ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 iops 2000 9. Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year. 10. Stable Testing random 4K reads EBS PIOPS+ SSD 11. Before PIOPS: After PIOPS: End-to-end latency 12. When should I use Provisioned IOPS? Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, and sequential IO access 13. IOPS vs. Throughput One input operation or block The network throughput available on the connection between EC2 and EBS The IOPS provisioned 14. IOPS vs. Throughput Bigger than 16k Greater than the mbps limit Greater than the IOPS provisioned 15. A little math Networks are in mbit/s, disks tend to be in MB/s. 1000mbit/s = 125 MB/s 500mbit/s = 67.5 MB/s Blocks/ IOs are in KB ~7,500 16KB blocks fit in 1000mbit/s ~30,000 4KB blocks fit in 1000mbit/s 16. Block Size 17. Queue Depth Maintain a number of pending I/O requests to get the most out of your Provisioned IOPS volume. The volumes must maintain an average queue length of 1 (rounded up to the nearest whole number) for every 200 provisioned IOPS in a minute 18. Performance Architecting for Performance Avoid IO saturation EBS Optimized Avoid throughput saturation RAID Achieving Consistent Performance Pre-warm PIOPS volumes Plan for snapshot Snapshot Performance 19. Architecting for Performance: Use EBS Optimized Instances 20. USING Provisioned IOPS Volumes Select a new type of Provisioned IOPS volumes SPECIFY ENOUGH IOPS!! Specify the volume capacity $ ec2-create-volume size 400 --availability-zone us-east-1b --type io1 iops 4000 21. Architecting for Performance: Avoid Throughput Saturation Example: Cluster Compute instance types have 2Gb/s bandwidth to EBS, more than 8 PIOPS volumes at 2000 IOPS each will saturate 2 Gb/s network EBS Optimized M3.2Xlarge instance has 1 Gb/s bandwidth dedicated to EBS, more than 16 PIOPS volumes at 500 IOPS each will saturate the 1 GB/s network 22. RAID RAID 10: provides increased redundancy Replace EBS volume without application downtime Increases read throughput However, 50% reduction of provisioned aggregate write performance E.g., MongoDB optimized around the benefits of RAID 10 RAID 0: All EBS volumes are replicated in the same AZ Increased throughput 23. Achieving Consistent Performance: Pre-warm PIOPS volumes There is a 5 to 50 percent performance reduction in IOPS when you first access the data on a PIOPS volume. Write to all blocks on volumes before first use $ dd of=/dev/md0 if=/dev/null 24. Snapshots Create snapshots (backups) of any Amazon EBS volume. The volume need not be attached to a running instance in order to take a snapshot. These snapshots can be used to create multiple new Amazon EBS volumes, expand the size of a volume, or move volumes across Availability Zones. The snapshots can be shared with specific AWS accounts or made public. You can use this functionality to increase the size of an existing volume, rapidly replicate development and testing environments, or use Snapshot Copy to copy snapshots to another region for disaster recovery or regional expansion. 25. Snapshot Performance To improve snapshot performance Increase the frequency of snapshots 26. Achieving Consistent Performance: Plan for Snapshot To minimize the impact of snapshots on performance of a master node: create snapshots from a read replica of your data Plan snapshots during off-peak usage Use cross-region snapshot copy to keep distant regional copies for HA / low latency local access 27. Performance Minimum production DB scale Always use Elastic Block Store (EBS) o Significant write cache o Great random IO performance o Enhanced durability compared to instance stores 28. Performance production scale Use Provisioned IOPS for all workloads targeting above 100 IOPS. Move up to higher bandwidth instance types (m1.xlarge, c1.xlarge, m2.4xlarge) Increase EBS volume size to >= 400 GB Increase number of volumes in RAID set 29. Performance Extra-large scale Leverage Cluster Compute instance types o More bandwidth to EBS o Ex. CC2 will make excellent primary nodes, particularly when paired with a large number of EBS volumes (= 8) Improve RAID configuration with: o effective_io_concurrency = # of stripes in RAID set 30. Amazon Provisioned IOPS o http://copperegg.com/amazon-provisioned-iops-ebs/ Benchmarking EBS performance: o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html Stratalux: Putting Amazons Provisioned IOPS to the test o http://www.stratalux.com/2012/08/09/putting-amazons-provisioned-iops-to-the-test/ Get started on EBS today! aws.amazon.com/ebs Questions: e-mail: [email protected] @milesward

Popular Tags:
of 30/30
Presenter Name Understanding Amazon EBS Availability and Performance Miles Ward Senior Manager, Solutions Architecture
Embed Size (px)