Let’s begin with the storage to which you are most likely already accustomed as a developer; that is, block storage.
Amazon EBS presents your data to your Amazon EC2 instance as a disk volume, providing the lowest-latency access to your data from single Amazon EC2 instances. Amazon EBS provides durable and persistent block storage volumes for use with Amazon EC2 instances. Each Amazon EBS volume is automatically replicated within its Availability Zone to protect your information from component failure, offering high availability and durability. Amazon EBS volumes offer the consistent and low-latency performance needed to run your workloads. With Amazon EBS, you can scale your usage up or down within minutes, while paying only for what you provision. Typical use cases for Amazon EBS include the following:
Amazon EBS is designed to achieve the following:
Amazon EBS volumes are 20 times more reliable than typical commodity disk drives, which fail with an AFR of around 4 percent.
Amazon EBS volumes persist independently from the running life of an Amazon EC2 instance. After a volume is attached to an instance, use it like any other physical hard drive. Amazon EBS volumes are flexible. For current-generation volumes attached to currentgeneration instance types, you can dynamically increase size, modify provisioned input/output operations per second (IOPS) capacity, and change the volume type on live production volumes without service interruptions.
Amazon EBS provides the following volume types, which differ in performance characteristics and price so that you can tailor your storage performance and cost to the needs of your applications.
SSD-backed: volumes Solid-state drive (SSD)–backed volumes are optimized for transactional workloads involving frequent read/write operations with small I/O size, where the dominant performance attribute is IOPS.
HDD-backed: volumes Hard disk drive (HDD)–backed volumes are optimized for large streaming workloads where throughput (measured in MiB/s) is a better performance measure than IOPS.
| SSD | HDD | |||
|---|---|---|---|---|
| General Purpose | Provisioned IOPS | Throughput Optimized | Cold | |
| Max volume size | 16 TiB | |||
| Max IOPS/volume | 10,000 | 32,000 | 500 | 250 |
| Max throughput/volume | 160 MiB/s | 500 MiB/s | 250 MiB/s |
| SSD | HDD | ||
|---|---|---|---|
| General Purpose | Provisioned IOPS | Throughput-Optimized | Cold |
|
|
|
Elastic Volumes is a feature of Amazon EBS that allows you to increase capacity dynamically, tune performance, and change the type of volume live. This can be done with no downtime or performance impact and with no changes to your application. Create a volume with the capacity and performance needed when you are ready to deploy your application, knowing that you have the ability to modify your volume configuration in the future and saving hours of planning cycles and preventing overprovisioning.
You can protect your data by creating point-in-time snapshots of Amazon EBS volumes, which are backed up to Amazon S3 for long-term durability. The volume does not need to be attached to a running instance to take a snapshot. As you continue to write data to a volume, periodically create a snapshot of the volume to use as a baseline for new volumes. These snapshots can be used to create multiple new Amazon EBS volumes or move volumes across Availability Zones. When you create a new volume from a snapshot, it is an exact copy of the original volume at the time the snapshot was taken.
If you are taking snapshots at regular intervals, such as once per day, you may be concerned about the cost of the storage. Snapshots are incremental backups, meaning that only the blocks on the volume that have changed after your most recent snapshot are saved, making this a cost-effective way to back up your block data. For example, if you have a volume with 100 GiB of data, but only 5 GiB of data have changed since your last snapshot, only the 5 GiB of modified data is written to Amazon S3. If you need to delete a snapshot, how do you know which snapshot to delete? Amazon EBS handles this for you. Even though snapshots are saved incrementally, the snapshot deletion process is designed so that you need to retain only the most recent snapshot to restore the volume. Amazon EBS will determine which dependent snapshots can be deleted to ensure that all other snapshots continue working.
Recall that Amazon EBS volumes are network-attached and not directly attached to the host like instance stores. On instances without support for Amazon EBS–optimized throughput, network traffic can contend with traffic between your instance and your Amazon EBS volumes. On Amazon EBS–optimized instances, the two types of traffic are kept separate. Some instance configurations incur an extra cost for using Amazon EBS–optimized, while others are always Amazon EBS–optimized at no extra cost.
For simplified data encryption, create encrypted Amazon EBS volumes with the Amazon EBS encryption feature. All Amazon EBS volume types support encryption, and you can use encrypted Amazon EBS volumes to meet a wide range of data-at-rest encryption requirements for regulated/audited data and applications. Amazon EBS encryption uses 256-bit Advanced Encryption Standard (AES-256) algorithms and an Amazon-managed key infrastructure called AWS Key Management Service (AWS KMS). The encryption occurs on the server that hosts the Amazon EC2 instance, providing encryption of data in transit from the Amazon EC2 instance to Amazon EBS storage.
You can encrypt using an AWS KMS–generated key, or you can choose to select a customer master key (CMK) that you create separately using AWS KMS. You can also encrypt your files prior to placing them on the volume. Snapshots of encrypted Amazon EBS volumes are automatically encrypted. Amazon EBS volumes that are restored from encrypted snapshots are also automatically encrypted.
To achieve optimal performance from your Amazon EBS volumes in a variety of scenarios, use the following best practices:
Use Amazon EBS-optimized instances The dedicated network throughput that you get when you request Amazon EBS–optimized support will make volume performance more predictable and consistent, and your Amazon EBS volume network traffic will not have to contend with your other instance traffic because they are kept separate.
Understand how performance is calculated When you measure the performance of your Amazon EBS volumes, it is important to understand the units of measure involved and how performance is calculated.
Understand your workload There is a relationship between the maximum performance of your Amazon EBS volumes, the size and number of I/O operations, and the time it takes for each action to complete. Each of these factors affects the others, and different applications are more sensitive to one factor or another.
On a given volume configuration, certain I/O characteristics drive the performance behavior for your Amazon EBS volumes. SSD-backed volumes, General-Purpose SSD, and Provisioned IOPS SSD deliver consistent performance whether an I/O operation is random or sequential. HDD-backed volumes, Throughput-Optimized HDD, and Cold HDD deliver optimal performance only when I/O operations are large and sequential.
To understand how SSD and HDD volumes will perform in your application, it is important to understand the connection between demand on the volume, the quantity of IOPS available to it, the time it takes for an I/O operation to complete, and the volume’s throughput limits.
Be aware of the performance penalty when initializing volumes from snapshots New Amazon EBS volumes receive their maximum performance the moment that they are available and do not require initialization (formerly known as pre-warming).
Storage blocks on volumes that were restored from snapshots, however, must be initialized (pulled down from Amazon S3 and written to the volume) before you can access the block. This preliminary action takes time and can cause a significant increase in the latency of an I/O operation the first time each block is accessed. Performance is restored after the data is accessed once.
For most applications, amortizing this cost over the lifetime of the volume is acceptable. For some applications, however, this performance hit is not acceptable. If that is the case, avoid a performance hit by accessing each block prior to putting the volume into production. This process is called initialization.
Factors that can degrade HDD performance When you create a snapshot of a Throughput-Optimized HDD or Cold HDD volume, performance may drop as far as the volume’s baseline value while the snapshot is in progress. This behavior is specific only to these volume types. Other factors that can limit performance include the following:
Increase read-ahead for high-throughput, read-heavy workloads Some workloads are read-heavy and access the block device through the operating system page cache (for example, from a file system). In this case, to achieve the maximum throughput, we recommend that you configure the read-ahead setting to 1 MiB. This is a per-block-device setting that should be applied only to your HDD volumes.
Use RAID 0 to maximize utilization of instance resources Some instance types can drive more I/O throughput than what you can provision for a single Amazon EBS volume. You can join multiple volumes of certain instance types together in a RAID 0 configuration to use the available bandwidth for these instances.
Track performance with Amazon CloudWatch Amazon CloudWatch, a monitoring and management service, provides performance metrics and status checks for your Amazon EBS volumes.
If you are using an Amazon EBS volume as a boot volume, your instance is no longer accessible, and you cannot use SSH or Remote Desktop Protocol (RDP) to access that boot volume. There are some steps that you can take, however, to access the volume. If you have an Amazon EC2 instance based on an Amazon Machine Image (AMI), you may just choose to terminate the instance and create a new one. If you do need access to that Amazon EBS boot volume, perform the following steps to make it accessible:
Amazon EC2 instance store is another type of block storage available to your Amazon EC2 instances. It provides temporary block-level storage, and the storage is located on disks that are physically attached to the host computer (unlike Amazon EBS volumes, which are network-attached).
If your data does not need to be resilient to reboots , restarts , or auto recovery , then your data may be a candidate for using instance store, but you should exercise caution.
Instance store should not be used for persistent storage needs. It is a type of ephemeral (short-lived) storage that does not persist if the instance fails or is terminated. Because instance store is on the host of your Amazon EC2 instance, it will provide the lowest-latency storage to your instance other than RAM. Instance store volumes can be used when incurring large amounts of I/O for your application at the lowest possible latency. You need to ensure that you have another source of truth of your data, however, and that the only copy is not placed on instance store. For data that needs to be durable, we recommend using Amazon EBS volumes instead. Not all instance types come with available instance store volume(s), and the size and type of volumes vary by instance type. When you launch an instance, the instance store is available at no additional cost, depending on the particular instance type. However, you must enable these volumes when you launch an Amazon EC2 instance, as you cannot add instance store volumes to an Amazon EC2 instance once it has been launched. After you launch an instance, the instance store volumes are available to the instance, but you cannot access them until they are mounted. Refer to the AWS documentation for Amazon EBS to learn more about mounting these volumes on different operating systems.
Many customers use a combination of instance store and Amazon EBS volumes with their instances. For example, you may choose to place your scratch data, tempdb , or other temporary fi les on instance store while your root volume is on Amazon EBS.
Do not use instance store for any production data.
With Amazon EC2, you can use both instance store–backed storage volumes and Amazon EBS–backed storage volumes with your instances, meaning you can have your instance boot off instance store; however, you would want this confi gured so that you are using an AMI and that new instances will be created if one fails. This is not recommended for your primary instances where it would cause an issue for users if the instance fails. This confi guration can save money on storage costs instead of using Amazon EBS as your boot volume in cases where your system is confi gured to be resilient to instances re-launching. It is critical to understand your application and infrastructure needs before choosing to use instance store-backed Amazon EC2 instances, so choose carefully.
Instance store–backed Amazon EC2 instances cannot be stopped and cannot take advantage of the auto recovery feature for Amazon EC2 instances. Some AWS customers build instances on the fl y that are completely resilient to reboot, relaunch, or failure and use instance store as their root volumes. This requires important due diligence regarding your application and infrastructure to ensure that this type of scenario would be right for you.