1

I am trying a scenario where cloud formation has to wait until an object is created in the specified bucket (where the object creation happens outside the scope of cloud formation by an external application).

I tried enabling bucket event notifications and hook a lambda function (so whenever an object is created in the bucket, lambda function is triggered). But I am not sure how to make cloud formation wait until this hooked lambda function execution is invoked.

Kindly let me if there any ideas on how to achieve this scenario.

Sasi
  • 83
  • 3
  • 14
  • 1
    How log do you have to wait? – Marcin May 07 '20 at 10:26
  • 1
    wait duration can be 40min – Sasi May 07 '20 at 10:27
  • 1
    How does the external process (that puts the object in the bucket) get triggered? Is it triggered from something in the CloudFormation template? – John Rotenstein May 07 '20 at 11:29
  • It can be manual PUT or an external CLI PUT (whereas in my case this object put is does by another application. – Sasi May 07 '20 at 11:32
  • 1
    Can you describe the complete flow? For example, does the CloudFormation template create the bucket? Should it then wait for a human to put an object in the bucket? Or can the object be pre-supplied? What triggers the creation of the object? Please edit your question and provide more information. – John Rotenstein May 08 '20 at 00:48

3 Answers3

0

Try using a wait condition to solve this: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-waitcondition.html

Chris Williams
  • 32,215
  • 4
  • 30
  • 68
  • thank you, @mokugo-devops. The part that I am unclear is the wait condition dependsOn field. Since the lambda function will be invoked only after object creation, I am not sure what needs to be filled in dependsOn field. I am not sure cfn-signal helps me here. – Sasi May 07 '20 at 10:39
  • That would be the resource it must create first i.e. the S3 bucket – Chris Williams May 07 '20 at 10:41
  • yes correct, but my wait has to depend on lambda to get invoked apart from S3 bucket, that is where i need help – Sasi May 07 '20 at 10:43
0

You could try using Custom CloudFormation resources: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-sns.html. This would require you can send a http request to a S3-url provided through an SNS notification.

You would create file based on the SNS-notification (using lambda?) and then send a request back to cloudformation.

Exelian
  • 5,749
  • 1
  • 30
  • 49
0

I think the following should work:

  1. Create WaitConditionHandle
  2. Create a lambda function and pass !Ref to the wait condition handle created as an environment variable. When you !Ref a wait condition you get an url address. The lambda has only one job - to call the url when invoked.
  3. Create WaitCondition and associate it with the wait handle created in step 1.
  4. Add DependsOn attribute to the WaitCondition so that the condition gets created after the last resource to be created before CFN should pause and wait.
  5. Use the S3 notification (as you already wrote in your question) to invoke lambda created in Step 2 when you get your object. Lambda gets invoked, calls the url, wait conditions stops waiting, and CFN should continue.

With the above there are no loops or long running processes, such as calling a lambda every 2 minutes.

Max timeout for the WaitCondition is 12 hours. You should adjust it 40 minutes or 1h for instance.

Marcin
  • 215,873
  • 14
  • 235
  • 294
  • thank you so much, interesting. I was able to understand till "calls the url, wait conditions stop". I got the waitconditionhandle url in lambda function, but not sure on how to call it. Can you help in "calls the url" step. – Sasi May 07 '20 at 11:06
  • @Sasi There are many ways, depending on your environment. One way for node js lambda is shown [here](https://stackoverflow.com/questions/51491872/how-to-call-rest-api-inside-aws-lambda-function-using-nodejs). Some techniques for python are [here](https://stackoverflow.com/questions/58994119/how-to-make-a-http-rest-call-in-aws-lambda-using-python). – Marcin May 07 '20 at 11:30