Things you should know before you start considering AWS EFS for your project
The main difference between EBS and EFS is that EBS is only accessible from a single EC2 instance in your particular AWS region, while EFS allows you to mount the file system across multiple regions and instances.
What use cases does Amazon EFS support?
Amazon EFS is designed to provide performance for a broad spectrum of workloads and applications, including Big Data and analytics, media processing workflows, content management, web serving, and home directories.
How is Amazon EFS different than Amazon S3?
Amazon EFS provides shared access to data using a traditional file sharing permissions model and hierarchical directory structure via the NFSv4 protocol. Applications that access data using a standard file system interface provided through the operating system can use Amazon EFS to take advantage of the scalability and reliability of file storage in the cloud without writing any new code or adjusting applications.
Amazon S3 is an object storage platform that uses a simple API for storing and accessing data. Applications that do not require a file system structure and are designed to work with object storage can use Amazon S3 as a massively scalable, durable, low-cost object storage solution.
How is Amazon EFS different than Amazon EBS?
Amazon EFS provides shared file storage for use with compute instances in the AWS Cloud and on-premises servers. Applications that require shared file access can use Amazon EFS for reliable file storage delivering high aggregate throughput to thousands of clients simultaneously.Amazon EBS is a cloud block storage service that provides direct access from a single Amazon EC2 instance to a dedicated storage volume. Applications that require persistent dedicated block access for a single host can use Amazon EBS as a highly available, low-latency block storage solution.
Amazon EFS’s distributed nature enables high levels of availability, durability, and scalability. This distributed architecture results in a small latency overhead for each file operation. Due to this per operation latency, overall throughput generally increases as the average I/O size increases, since the overhead is amortized over a larger amount of data. Amazon EFS’s support for highly parallelized workloads (i.e. with concurrent operations from multiple threads and multiple EC2 instances) enables high levels of aggregate throughput and IOPS.
Amazon EFS is a file storage service for use with Amazon EC2. Amazon EFS provides a file system interface, file system access semantics (such as strong consistency and file locking), and concurrently-accessible storage for up to thousands of Amazon EC2 instances.
To access your file system, you mount the file system on an Amazon EC2 Linux-based instance using the standard Linux mount command and the file system’s DNS name. Once mounted, you can work with the files and directories in your file system just like you would with a local file system.
Storage classes and lifecycle management
Amazon EFS offers a Standard and an Infrequent Access storage class. The Standard storage class is designed for active file system workloads and you pay only for the file system storage you use per month. Amazon EFS Infrequent Access (Amazon EFS IA) is a lower-cost storage class that’s cost-optimized for files not accessed every day. Data stored on the Amazon EFS IA storage class costs 85% less than Standard and you pay a fee each time you read from or write to a file. Amazon EFS file systems transparently serve data from both storage classes.
When reading from or writing to Amazon EFS IA, your first-byte latency is higher than that of Amazon EFS Standard. Amazon EFS Standard is designed to provide single-digit latencies on average, and Amazon EFS IA is designed to provide double-digit latencies on average.
Scale and performance
Amazon EFS file systems can store petabytes of data. Amazon EFS file systems are elastic and automatically grow and shrink as you add and remove files. You do not provision file system size upfront, and you pay only for what you use.
Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow elastically to petabyte-scale and allowing massively parallel access from Amazon EC2 instances to your data. Amazon EFS’s distributed design avoids the bottlenecks and constraints inherent to traditional file servers.
Amazon EFS’s distributed nature enables high levels of availability, durability, and scalability. This distributed architecture results in a small latency overhead for each file operation. Due to this per operation latency, overall throughput generally increases as the average I/O size increases, since the overhead is amortized over a larger amount of data. Amazon EFS’s support for highly parallelized workloads (i.e. with consistent operations from multiple threads and multiple EC2 instances) enables high levels of aggregate throughput and IOPS.
Access from Lambda functions
You can access Amazon EFS from functions running in AWS Lambda by referencing an Amazon EFS file system in your function settings. Getting started instructions can be found in the AWS Lambda documentation.