1Cloud Compliance in NIST CloudCheckr 2017

Amazon EBS:An Essential Guide to Cost andPerformance Optimization

WHITEPAPER

AWS Elastic Block Store (EBS) is Amazon’s most widely used cloud storage service—and with good reason. It is designed specifically for use with EC2, is suited to a wide variety of use cases, offers four different volume types, and boasts an extensive range of options and features. However, it also represents a significant chunk of your monthly AWS bill. So, it’s essential you use EBS wisely by striking the right balance between cost and performance, and eliminating unnecessary waste on storage resources. To make the most of your organization’s cloud spend and leverage resources efficiently, it’s important to understand Amazon EBS. This guide will help you by:

• Explaining the four different EBS volume types from a cost and performance perspective

• Discussing some of the hidden costs of EBS

• Providing a few monitoring tips to help you reduce EBS expenditure

But first let’s take a quick recap of what Amazon EBS actually is.

Abstract

EBS is one of Amazon’s two core block storage options designed for use with EC2 instances. It is gaining increasing favor over ephemeral block storage alternative Instance Store. This is because, unlike instance store, data stored on EBS is persistent, allowing you to stop and restart instances without loss of data.

What Is Amazon EBS?

2Cloud Compliance in NIST CloudCheckr 2017

This can help you reduce your cloud costs, as you can shut down instances whenever applications aren’t needed. What’s more, it offers faster boot times and a wider range of features. In line with block-level storage in general, EBS comes in the form of mountable fixed-sized raw storage disks or volumes, which you can format with the file system of your choice. Although you can only dedicate an EBS volume to one instance at any one time, you can attach more than one volume to any individual instance.

As the alternative block storage system to Instance Store, EBS shares many similar characteristics with its ephemeral counterpart. However, it is significantly different from Amazon’s object storage service, S3. EBS gives developers a way to format volumes with a familiar file system, so they can build application environments in the cloud based on traditional on-premise architecture. By contrast, S3 is general-purpose storage geared towards applications that are either born in or rearchitected for the cloud. You can only access EBS through an EC2 instance, whereas objects stored on S3 are available to anyone anywhere, provided they have the object path name or access via API. But one of the most significant differences is cost. EBS is more expensive than S3. What’s more, unlike S3, it is provisioned storage. So make sure you pay careful attention to your volume sizes to avoid paying for storage you don’t need. This is particularly important at the outset, as there’s no easy way to resize volumes on the fly without interruption to availability. However, with so many applications designed for use with block storage, EBS is suited to a much wider range of use cases, including both relational and NoSQL databases and enterprise applications such as SAP and Microsoft Exchange.

How Does It Compare with S3?

Amazon’s New Core Storage OptionIn June last year, AWS launched a new fully managed storage service, Elastic File System (EFS), for use with applications built upon a file system architecture. As with EBS, it is designed for use with EC2. However, as a network file storage system, it differs from block storage in a number of important ways. Unlike EBS volumes, file systems can be shared by any number of instances in the same Amazon region. Storage is based on a highly scalable distributed design and can grow to petabyte levels of scale, making it suitable for massively parallel workloads that share a common file system. Although it’s more expensive than EBS, just like S3, it scales automatically—so you only pay for what you actually use.

3Cloud Compliance in NIST CloudCheckr 2017

EBS offers a raft of useful features, some of which are included in the price while others come at an additional cost:

Key EBS Features

Snapshots

Dedicated Network Capacity (EBS Optimization)

Data Encryption

Automatic Replication

Snapshots are point-in-time backups of your EBS volumes to S3. Backups are incremental, which means only the blocks that have changed since your last snapshot are saved. Not only that, but snapshots also leverage data compression and the lower storage charges on S3, making it a cost-effective method of backing up EBS volumes. You can also use EBS snapshots to create template machine images.

EBS Optimization provides dedicated bandwidth between your EC2 instances and EBS volumes. It is designed to improve I/O and throughput performance by minimizing contention between competing processes that share EC2 resources. All instances in the C4, D2, I3, M4, P2, R4 and X1 families are optimized by default. EBS Optimization is also available at an additional charge to selected instances in the C3, G2, M3 and R3 families.

EBS supports data encryption to AES 256-bit, which can help users to meet compliance standards, such as HIPAA, PCI and NIST. You can encrypt data at rest inside the volume, all snapshots created from the volume, and all disk I/O. It is available to certain EC2 instance types and comes at no additional cost.

AWS automatically replicates your EBS volumes within the same Availability Zone (AZ). This helps prevent data loss in the event of hardware failure. However, this may not be sufficient to safeguard your data at enterprise scale, as the frequency of volume failures within a large cloud environment will inevitably be higher. What’s more, this doesn’t protect you if an entire AZ goes down; ultimately, your EBS costs will also include the storage needed for your failover architecture—either across AZs or regions, depending on your availability requirements.

4Cloud Compliance in NIST CloudCheckr 2017

Amazon offers a choice of four different categories of EBS volume. As each option is tailored to different types of workload, it’s important to understand the differences between them to get the best balance of cost and performance.

EBS Volume Types

Solid State Drives (SSD)

SSDs are capable of delivering substantially higher IOPS (I/O operations per second) than traditional magnetic drives and perform best when handling very frequent but small transactions. However, they cost significantly more, at more than twice the price of hard disk drives.

EBS General Purpose SSDVolume size: 1GB–16TBBaseline IOPS: 3 IOPS per GB up to maximum of 10,000Burst I/O: Up to 3,000 (applicable to volumes up to 1TB)I/O: Included in the price

The default EBS option designed for general use, including low-latency transactional applications, dev/test environments and boot volumes. Provisioned storage costs $0.10 per GB-month in the US East (N. Virginia region).

EBS Provisioned IOPS SSDVolume size: 4GB–16TBI/O: $0.065 per provisioned IOPS-monthThe storage class with the highest performance capabilities of all four EBS volume types. But also the most expensive at $0.125 per GB-month of provisioned storage in the US East (N. Virginia region). On top of that, you pay separately for provisioned IOPS (PIOPS). This is something that’s included in the price of General Purpose SSD volumes. So, at lower levels of PIOPS, you can actually get better performance from General Purpose volumes while also paying a lower price. One way to avoid making such mistakes is to monitor your AWS deployments using third-party tools that help you align your EBS storage with best practices. In terms of use cases, Provisioned IOPS SSD is particularly well suited to NoSQL frameworks such as Apache Cassandra and HBase and I/O-intensive relational databases.

5Cloud Compliance in NIST CloudCheckr 2017

Hard Disk Drives (HDD)

EBS Snapshot Costs

HDDs are both less expensive and less responsive than SSDs, with considerably lower IOPS performance. However, they do offer slightly better throughput levels, making them ideal for applications that read and write large chunks of data.

As EBS snapshots are incremental, each individual backup is relatively small. However, this means snapshot costs can creep up slowly—until, before you know it, your charges have spiralled out of control. This can be a particular problem if you automate your snapshots, where it’s essential to build in a deletion schedule for older backups.

Snapshots are backed up to S3, but show up in the EBS section of your monthly AWS bill

EBS Throughput Optimized HDDVolume size: 500GB–16TBBaseline throughput per volume: 40MB/s per TB up to a maximum of 500MB/sBurst throughput: Up to 500 MB/s (applicable to volumes up to 2TB)I/O: Included in the price

A low-cost general magnetic storage option for use as data volumes, providing the best throughput capabilities of all EBS volume types along with throughput bursting based on a usage credit system. Provisioned storage costs $0.045 per GB-month (N. Virginia) and use cases include log processing, data warehousing, and big data.

EBS Cold HDDVolume size: 500GB–16TBBaseline throughput per volume: 12MB/s per TB up to a maximum of 250MB/sBurst throughput: Up to 250 MB/s (applicable to volumes up to 3.125TB)I/O: Included in the price

The least expensive EBS option at $0.025 per GB-month of provisioned storage (N. Virginia). Cold HDD is designed to offer a cost-orientated block storage option for applications that require only occasional or unpredictable access.

6Cloud Compliance in NIST CloudCheckr 2017

Backup Size The monthly cost of storing your snapshots is dictated by the size of your initial backup and the total size of your incremental backups over your retention period. Your initial backup size will be equal to the amount of storage actually used on your EBS volume, with a slight reduction to allow for data compression. The total size of your incremental backups will depend on how frequently the data on your volume changes over time and how long you choose to retain your snapshots. Data on a production server typically changes by about 3% per day. Multiply that by 30 days, and you arrive at a figure of 90% per month. So, based on daily snapshots over a retention period of 30 days, you’ll likely need about twice the size of your initial snapshot to accommodate both your first and subsequent backups. Pricing for EBS snapshot storage is slightly different from standard S3 rates at $0.05 per GB-month (N. Virginia).Beware of Unattached Volumes and Orphan SnapshotsWhen you terminate an EC2 instance, by default, only EBS root device volumes are automatically deleted. By contrast, attached volumes and snapshots persist on EBS and S3 respectively. To avoid running up unnecessary storage costs, you should delete unattached volumes and orphan snapshots when you no longer need them.

Note: make sure you tag EBS resources so you can keep track of them throughout their lifetime. You can also simplify the task of maintaining your EBS inventory by using third-party cloud management tools that automatically clean up unused resources for you.

EBS Monitoring Tips

I/O and Throughput Utilization

Amazon CloudWatch provides a number of standard metrics for checking the operational behavior of your EBS volumes. The most important of these are:

• VolumeReadOps: Total number of read operations per specified period

• VolumeWriteOps: Total number of write operations per specified period

• VolumeThroughputPercentage (Provisioned IOPS SSD only): IOPS delivered as a percentage of total IOPS provisioned for volume

In the case of IOPS, CloudWatch doesn’t provide a metric directly. So you’ll need to calculate it yourself. By default, CloudWatch collects data every five minutes. So you’d need to add together your read and write metrics and divide by 300 seconds as follows:

IOPS = (VolumeReadOps + VolumeWriteOps) / 300

Your IOPS and throughput percentage metrics will help you determine whether you’re under- or overutilizing your EBS volumes. Using baseline IOPS, baseline throughput and bursting capabilities as a guide, you may need to resize them to get a better balance of cost and performance. Alternatively, you could consolidate a set of smaller attached volumes into fewer larger ones or consider switching to another volume type. With Provisioned IOPS SSD in particular, you should pay close attention to your IOPS utilization, as you pay for your provisioned IOPS directly.

7Cloud Compliance in NIST CloudCheckr 2017

Disk Utilization

EBS Best Practices

Comprehensive Cloud Management to Optimize Cloud Spend

Disk utilization is not a standard metric that you can monitor in CloudWatch. So, it’s not easy to check whether you’re making good use of the storage you’ve actually provisioned. Nevertheless, it’s still possible to monitor disk space as a custom metric by using a CloudWatch monitoring script, which you can download from the AWS sample code library. Again, with Provisioned IOPS SSD, it’s particularly important to monitor your disk utilization as, unlike with the other three volume types, you can scale storage independently of IOPS, giving you more scope for precise optimization.

Best practices, such as tagging resources, monitoring utilization, and cleaning up unattached volumes and orphan snapshots, are key to getting a bigger bang for your buck from your EBS deployments. You should also minimize the time-consuming and costly manual processes involved in maintaining your EBS storage. For example, you could implement a solution that automates your snapshot schedule, saving you time and ensuring you never miss a backup. Finally, you should integrate migration processes into your storage lifecycle, such as rotating instance logs from EBS to S3 or Glacier. By offloading this data to less expensive storage, you can make even further savings on your monthly cloud bills.

Savvy enterprises have realized that managing a rapidly scaling environment in the cloud can be a struggle. Cloud management platforms like CloudCheckr can help simplify and streamline the process of optimizing workloads in a dynamic, ephemeral environment. With CloudCheckr, organizations gain visibility and control across resources, with self-healing automation for critical tasks to keep infrastructure safe and stable.

The CloudCheckr cloud management platform unifies cost, security, and inventory management with visibility and intelligence to mitigate security risks, optimize costs, and increase operational efficiencies across cloud infrastructure. With continuous monitoring, 400 best practice checks, and built-in automation, CloudCheckr enables IT, Security, and Finance teams to manage their AWS environments with confidence. Government organizations and Global 2000 enterprises trust CloudCheckr to unify their native AWS data and deliver the most robust cloud management platform in today’s marketplace.

About CloudCheckr

VISIT US ONLINE