0

I'm using CodePipeline to deploy my CloudFormation templates that contain Lambda functions as AWS::SAM::Functions.

The CodePipeline is triggered by a commit in my main branch on GitHub.

The Source Stage in the CodePipeline retrieves the source files from GitHub. Zero or more Lambda functions could change in a commit. There are several Lambda Functions in this repository.

I intend on running through taskcat for CloudFormation Templates and Unit Tests for Lambda Python code during a test stage and then deploy the CloudFormation templates and Lambda Functions to production. The problem is, I can't figure out how to differentiate between changed and unchanged Lambda functions or automate the deployment of these Lambda functions.

I would like to only test and deploy new or update changed Lambda functions along with my CloudFormation templates - what is the best practice for this (ideally without Terraform or hacks)?

Jerezle
  • 43
  • 4

1 Answers1

2

Regarding testing: Best practice is actually to simply test all lambda code in the repo on push before deploying. You might skip some work for example with github actions that you only test the files that have changed, but it definitely takes some scripting and it hardly ever adds much value. Each testing tool has its own way of dealing with that (sometimes you can simply pass the files you want to test as an argument and then its easy, but sometimes test tools are more of a all-or-nothing approach and it gets quite complicatedreal fast).

Also, personally I'm not a big fan of taskcat since it doesn't really add a lot of value and it's not a very intuitive tool (also relatively outdated IMO). Is there a reason you need to do these types of testing?

Regarding deployment: There are a few considerations when trying to only update lambdas that have changed.

Firstly, cloudformation already does this automatically: as long as the cloudformation resource for the lambda doesn't change, the lambda will not be updated.

However, SAM has a small problem there, since it will re-package the lambda code on every pipeline run and update the CodeUri property of the lambda. And thus the lambda gets updated (even though the code might stay the same).

To work around this, you have several options:

  1. Simply accept that SAM updates your function even though the code might not have changed.
  2. Build SAM locally, and use the --cached and --cache-dir option when deploying in your pipeline. Make sure to push the folder that you set as cache-dir.
  3. Use a different file packaging tool than SAM. Either some custom script that or something else that only pushes your code to s3 when the files have changed.

If you're into programming I'd suggest you take a look into CDK. It's a major upgrade from cloudformation/SAM, and it handles code bundling better (only updates when files have changed). Also the testing options are much wider for CDK.

LRutten
  • 1,634
  • 7
  • 17
  • Thanks for the info! I'm checking out the CDK right now - I have a coding background so this looks like a great option. One of my bigger issues is that I have no idea which Lambda functions have changed during the source stage in CodePipeline. Is there anyway to get a diff or something from the Source Artifacts? – Jerezle Sep 08 '21 at 09:26
  • Funnily enough I just answered a similar question about that here: https://stackoverflow.com/questions/69090695/find-which-files-were-updated-in-a-commit-in-aws-codepipeline?noredirect=1#comment122125164_69090695. Is this of any help? Also look in the comments for more info. – LRutten Sep 08 '21 at 11:54