0

All my terraform planning since ever triggers a replacement of a dependency code Lambda Layer on every push even with my poetry.lock file not changing. Is there a way to create a folder that will always look the same — have the same base64sha256 — when downloading Python dependencies?

To frame the issue better, this is what I always get on terraform plan. If you also had this problem this will look familiar:

  # aws_lambda_layer_version.dependencies_layer must be replaced
+/- resource "aws_lambda_layer_version" "dependencies_layer" {
      ~ arn                         = "arn:aws:lambda:us-west-2:REDACTED:layer:REDACTED:74" -> (known after apply)
      ~ created_date                = "2021-09-23T22:20:17.910+0000" -> (known after apply)
      ~ id                          = "arn:aws:lambda:us-west-2:REDACTED:layer:REDACTED:74" -> (known after apply)
      ~ layer_arn                   = "arn:aws:lambda:us-west-2:REDACTED:layer:REDACTED" -> (known after apply)
      + signing_job_arn             = (known after apply)
      + signing_profile_version_arn = (known after apply)
      ~ source_code_hash            = "Af3BfEOzwufMnD8x+yYk8v8FoqjiuP2C9jrLtS3m/nk=" -> "AzkTkha5+6cD1R9ppYv6k8vJA7FDGnCJ33gblm+jS8U=" # forces replacement
      ~ source_code_size            = 15363733 -> (known after apply)
      ~ version                     = "74" -> (known after apply)

I managed to fix the main source code lambda replacement with the help of the Terraform lambda source_code_hash update with same code question and applied it to this Lambda Layer. But had no success:

data "archive_file" "dependencies_layer" {
  type        = "zip"
  source_dir  = "../build/dependencies_layer/"
  output_path = "../build/dependencies_layer.zip"
}


resource "aws_lambda_layer_version" "dependencies_layer" {
  layer_name          = "${var.service_name}-${var.stage}-dependencies-layer"
  description         = "Python requirements lambda layer for ${var.service_name} ${var.stage} stage."
  compatible_runtimes = ["python3.9"]
  filename            = data.archive_file.dependencies_layer.output_path
  source_code_hash    = data.archive_file.dependencies_layer.output_base64sha256
}

This makes me think that the creation of that folder is not downloading (and building?) the same files all the time. How do you manage to make this to be a hashable .zip? Are there some files that can be deleted that are making this happen? Today my creation of the dependency_layer folder is quite simple and looks like this:

build/requirements.txt: build poetry.lock
    poetry export -f requirements.txt --output build/requirements.txt

build/dependencies_layer: build build/requirements.txt
    PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 poetry run pip install -r build/requirements.txt -t build/dependencies_layer/python/lib/python3.9/site-packages/ --upgrade -q

Edit: For clarification, the dependencies are locked with == versioning:

[tool.poetry.dependencies]
python = "~3.9"
python-dateutil = "==2.8.2"
fastapi = "==0.68.1"
dependencies = "==7.1.0"
mangum = "==0.12.2"
aiodynamo = {version = "==21.9", extras = ["aiohttp"]}
pydantic = "==1.8.2"
aiohttp = "==3.7.4"
yarl = "==1.6.3"

Edit 2: When I apply this plan it replace the layer but at the AWS console it do not save the old version. So it adds the 74 but there is no 73 after the apply.

João Filipe Storarri
  • 1
  • 3
  • do you have minor version pinning in place/exact versions set? – Paul Collingwood Sep 24 '21 at 12:53
  • Yes, @PaulCollingwood. All my dependencies are managed with `==` locks and this happens on deploys in the same 4 minutes range, so I don't think someone managed to update libs all the times. I will put the dependencies on the question. – João Filipe Storarri Sep 24 '21 at 13:15
  • Please see my answer here https://stackoverflow.com/questions/73671437/only-create-new-archive-file-for-lambda-when-code-changes/73676194#73676194 – Leslie Alldridge Sep 12 '22 at 07:13

0 Answers0