6

I'm new to Cloudwatch events and to Fargate. I want to trigger a Fargate task (Python) to run whenever a file is uploaded to a specific S3 bucket. I can get the task to run whenever I upload a file, and can see the name in the event log; however I can't figure out a simple way to read the event data in Fargate. I've been researching this the past couple of days and haven't found solution other than reading the event log or using a lambda to invoke the task and to put the event data in a message queue.

Is there a simple way to obtain the event data in Fargate with boto3? It's likely that I'm not looking in the right places or asking the right question.

Thanks

J. O'Ryan
  • 133
  • 1
  • 6

2 Answers2

8

One of the easiest options that you can configure is two targets for same s3 image upload event.

  • Push the Same Event to SQS
  • launch Fargate task at the same time

Read Message Event from SQS when Fargate is up (No Lambda in between), also same task definition that will work a normal use case, make sure you exit the process after reading the message from sqs.

So in this case whenever Fargate Task up, it will read messages from the SQS. enter image description here

Adiii
  • 54,482
  • 7
  • 145
  • 148
  • Hi @Adiii how would this work in the event of multiple items in the SQS queue at the same time? :) – Chris Williams Jul 05 '20 at 18:59
  • Thanks Adiii, this is a good solution that will work with my use case. – J. O'Ryan Jul 06 '20 at 12:59
  • 2
    @ChrisWilliams I do not think it will create an issue, the task can read all available messages in the Queue and exit if no more message,by doing this it will decrease launch overhead time that will be required for each event, as primary purpose is to process event and grab required data from the event. – Adiii Jul 06 '20 at 15:51
  • 1
    Sure thing, great answer by the way :) – Chris Williams Jul 06 '20 at 15:53
  • 1
    Thanks @ChrisWilliams :) – Adiii Jul 06 '20 at 15:55
2

To do this you would need to use a input transformer.

Each time a event rule is triggered a JSON object accessible to use for in the transformation.

As the event itself is not accessible within the container (like with Lambda functions), the idea is that you would actually forward key information as environment variables and manipulate in your container.

At this time it does not look like every service supports this in the console so you have the following options:

You can view a tutorial for this exact scenario from this link.

Chris Williams
  • 32,215
  • 4
  • 30
  • 68