7

I'm running an apache webapp on a container in ecs fargate, but I need to be able to persist certain files so that any changes to them are available to all users through all versions of the webapp. Unfortunately, fargate's support for EFS is in-development, so I need a persistent storage solution at least until that's useable (or until a release date for that feature is announced so I can just say I'll deal with this then).

I tried to switch over from fargate to ec2, as EFS is supposedly compatible with ec2, but I couldn't figure out a way to get past the "unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster." error. I tried all solutions in AWS ECS Error when running task: No Container Instances were found in your cluster , as well as everything I could find in a few google searches, but to no avail.

The way the files are accessed is essentially:

  1. A php file calls the database
  2. database points to the location on the webapp that the application data is stored
  3. data is read / written to the file in the webapp

does anyone know of a persistent storage solution for ecs fargate? And how would I go about setting it up?

Brandon
  • 111
  • 1
  • 2
  • 6
  • How about storing your files in S3 rather than locally? – Ben Whaley Oct 29 '19 at 20:53
  • I'm not too familiar with S3 (and aws. and docker.), so I'm not sure. would I be able to upload the files to S3, and have multiple users read/write from them? Would I be able to redirect user requests from the webapp to S3? How would I go about setting it up? etc. – Brandon Oct 29 '19 at 21:19
  • It all depends on your use case a bit. If multiple users need to read/write simultaneously, e.g. a Google Docs-type collaboration, it probably needs to be in memory. If you're able to lock the file for editing by one user at a time, I'd probably persist the file to S3 and use a dynamodb lock to control access. – Ben Whaley Oct 29 '19 at 23:39
  • Fargate is stateless, like Lambda; any storage is ephemeral. If possible, you may wish to reconsider rearchitecting your app to be stateless, and able to run on Fargate. If its not possible or practical you’ll have to use ECS, which EFS works nicely with. Using EFS with ECS-EC2 is well documented and there’s a great tutorial here https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_efs.html. – hephalump Oct 30 '19 at 00:26

3 Answers3

9

EDIT : Fargate now supports EFS as persistent storage. So you don't need to use something like S3FS or any other hacky solution :

https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-ecs-aws-fargate-support-amazon-efs-filesystems-generally-available/

Original answer (not relevent any more):

Fargate does not support persistent storage at the moment, because you don't have access to the underlying infrastructure, and as such, you cannot mount an EFS volume. And it might not happen any time soon.

But one solution would be to use S3FS, which mounts an S3 bucket as a file system, directly on the container, so you don't need to have access to the infrastructure.

You can find it here : https://github.com/s3fs-fuse/s3fs-fuse

The project is popular and well maintained. It should work, but keep in mind that it's a bit of a hacky solution. It's not a fully POSIX compliant solution, but it's OK for storing simple files. Performance will also be degraded compared to a local file system. If you're serving the stored files, it's better to add a CDN on top, with optimized caching.

kristianp
  • 5,496
  • 37
  • 56
Hakro
  • 2,625
  • 1
  • 12
  • 23
6

Different horse for different courses, Fargate is designed to zero management of infrastructure which means just specify the Docker image and then leave everything on AWS.

Bind mount host volumes are supported when using either the EC2 or Fargate launch types. Fargate tasks only support nonpersistent storage volumes, so the host and sourcePath fields are not supported.

The work arround is volume mounts.

Fargate Task Storage

When provisioned, each Fargate task receives the following storage. Task storage is ephemeral. After a Fargate task stops, the storage is deleted.

10 GB of Docker layer storage

An additional 4 GB for volume mounts. This can be mounted and shared among containers using the volumes, mountPoints and volumesFrom parameters in the task definition.

To provide nonpersistent empty storage for containers in a Fargate task

In this example, you may have two database containers that need to access the same scratch file storage location during a task.

In the task definition volumes section, define a volume with the name database_scratch.

  "volumes": [
    {
      "name": "database_scratch",
      "host": {}
    }
  ]

In the containerDefinitions section, create the database container definitions so they mount the nonpersistent storage.

  "containerDefinitions": [
    {
      "name": "database1",
      "image": "my-repo/database",
      "cpu": 100,
      "memory": 100,
      "essential": true,
      "mountPoints": [
        {
          "sourceVolume": "database_scratch",
          "containerPath": "/var/scratch"
        }
      ]
    },
    {
      "name": "database2",
      "image": "my-repo/database",
      "cpu": 100,
      "memory": 100,
      "essential": true,
      "mountPoints": [
        {
          "sourceVolume": "database_scratch",
          "containerPath": "/var/scratch"
        }
      ]
    }
  ]

If you are looking for a way to bind host volume then you are expecting wrong from fargate as there is no host in case of fargate.

You need the Ec2 type ECS Task.

To provide persistent storage for containers using a bind mount

When using bind mounts, if a sourcePath value is specified the data persists even after all containers that referenced it have stopped. Any files that exist at the sourcePath are presented to the containers at the containerPath value, and any files that are written to the containerPath value are written to the sourcePath value on the container instance.

In the task definition volumes section, define a bind mount with name and sourcePath values.

  "volumes": [
    {
      "name": "webdata",
      "host": {
        "sourcePath": "/ecs/webdata"
      }
    }
  ]

In the containerDefinitions section, define a container with mountPoints values that reference the name of the defined bind mount and the containerPath value to mount the bind mount at on the container.

  "containerDefinitions": [
    {
      "name": "web",
      "image": "nginx",
      "cpu": 99,
      "memory": 100,
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        }
      ],
      "essential": true,
      "mountPoints": [
        {
          "sourceVolume": "webdata",
          "containerPath": "/usr/share/nginx/html"
        }
      ]
    }
  ]

bind-mounts

again

The host and sourcePath parameters are not supported for Fargate tasks.

Adiii
  • 54,482
  • 7
  • 145
  • 148
  • I was able to get persistent storage working on EC2 after some finagling. I have a few questions about this solution: 1. is it possible to access/download the mounted files? Say, if I needed to back up the files, or access them outside of a container, would I be able to? 2. am I correct in assuming the mounts can be accessed by multiple containers at the same time? 3. Is it possible to mount the files across regions? Or is it limited to the region the volume was created in? – Brandon Oct 31 '19 at 14:00
  • 1. yes you can download file from ec2 and you will able to see mounted files after doing SSH to ec2 instance. 2) yes it the same volume attach with multiple container you can access. 3) the third one seems complicated, I think volume not allowing access cross region, you need snapsshot. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html – Adiii Oct 31 '19 at 14:12
  • I sshed into the instances, and found the folders I pointed to in the .json. However, for some reason those folders are empty. I believe that the persistent storage worked, as I had stopped the task (and set number of tasks to 0), waited 17 hours, launched them again, and confirmed that the changes from post-launch were still in effect. What could cause the folders I mounted the volume on to be empty? – Brandon Oct 31 '19 at 15:40
  • I figured it out. I have permissions set in the container correctly, but the ec2 instance does not have the correct permissions to create new files. – Brandon Nov 01 '19 at 15:52
5

Fargate nows supports EFS. I've personally set up an app that uses it and I can say that it works, in general, but the performance is terrible. So really, it depends on what kind of I/O performance you need.

Specifically, I found that EFS has a hard time listing files in a directory with thousands of files, as well as may be slow when reading the files' contents.

There is a lot of content regarding EFS performance out there, so make sure you look into it. However, you may now finally have your solution for file persistence in Fargate!

Y L
  • 51
  • 1
  • 3