1

I have an ECS Service with Tasks using an EFS Volume. I also have a Datasync task to update the files on the EFS Filesystem(destination) from an S3 Bucket(source)

The container task is serving files stored on the EFS.

Whenever I run the Datasync task and update EFS the data is not updated on the container. In order to have the latest data I have to set the desired tasks on the Service to 0 and then to 1 again so it creates new instances of the task.

I want to avoid having downtime for my service. Is there a way to have the latest data without doing what I previously explained?

Update:

Looks like my main issue is that the container is indexing files to memory so a restart(redeployment is needed). But I still have an availability issue, I need a way to automate this process

arbermejo
  • 106
  • 5
  • It sounds like your task is caching the data in memory after it reads it from disk. – Mark B May 31 '22 at 21:02
  • In that case, how to clean the clean memory if I am not building the image? – arbermejo May 31 '22 at 21:09
  • This is entirely based on the software running in your container. It's not really related to AWS at all. You need to understand how the software in your container works in order to know how to clear the cache/memory. You have provided no information at all about that software, so nobody will be able to help you unless you expand your question quite a bit. It could be that the only way to clear that is to replace the container, just like you are doing now. – Mark B Jun 01 '22 at 11:21

1 Answers1

0

Ok so after a lot of research I found out that there's an AWS-CLI command to force a new deployment for the ECS Service, with that new tasks are created and the old ones are kept until the newest ones are in a stable state, that removes my availability issue.

The command is:

aws ecs update-service --force-new-deployment --service my-service-name --cluster my-cluster-name

Now I needed a way to automatically update my task whenever the Datasync task is executed, I thought about creating a pipeline for it to execute the command from a Codebuild container. The issue is that CodePipeline does not have a Datasync Task Execution as a trigger for the Source Stage...

My solution was to create an AWS EventBridge Rule with a Pattern to detect the event of Datasync Task State Change for my task and then I added as a Target a CodeBuild Project where I defined the previously stated command in the buildspec

The pattern looks like this in case someone needs it (wasn't able to find any example about it online)

{
  "detail-type": ["DataSync Task State Change"],
  "resources": ["arn:aws:datasync:us-east-1:1234567890:task/task-1234567890"],
  "source": ["aws.datasync"],
  "detail": {
    "State": ["AVAILABLE"]
  }
}

With that my Datasync Task Completes its execution the EventBridge rule will automatically trigger my CodeBuild project, therefore automatically updating my ECS Servcice with 0 downtime!

arbermejo
  • 106
  • 5