1

AWS Lambda uploading requires the generation of a zip archive of required source code and libraries. For use of NodeJS as the language for Lambda, it may be more typically the case that you want a source file and the node_modules directory to be included in the zip archive. The Terraform archive provider gives a file_archive resource which works well when it can be used. It can't be used when you want more than just 1 file or 1 directory. See feature request . To work around this, I came up with this code below. It executes steps but not in the required sequence. Run it once and it updates the zip file, but doesn't upload it to AWS. I run it again and it uploads to AWS. 

# This resource checks the state of the node_modules directory, hoping to determine,
# most of the time, when there was a change in that directory. Output
# is a 'mark' file with that data in it. That file can be hashed to
# trigger updates to zip file creation.
resource "null_resource" "get_directory_mark" {
    provisioner "local-exec" {
        command     = "ls -l node_modules > node_modules.mark; find node_modules -type d -ls >> node_modules.mark"
        interpreter = ["bash", "-lc"]
    }

    triggers = {
        always = "${timestamp()}" # will trigger each run - small cost.
    }
}

resource "null_resource" "make_zip" {
    depends_on = ["null_resource.get_directory_mark"]

    provisioner "local-exec" {
        command     = "zip -r ${var.lambda_zip} ${var.lambda_function_name}.js node_modules"
        interpreter = ["bash", "-lc"]
    }

    triggers = {
        source_hash  = "${sha1("${file("lambda_process_firewall_updates.js")}")}"
        node_modules = "${sha1("${file("node_modules.mark")}")}"                  # see above
    }
}

resource "aws_lambda_function" "lambda_process" {
    depends_on       = ["null_resource.make_zip"]
    filename         = "${var.lambda_zip}"
    function_name    = "${var.lambda_function_name}"
    description      = "process items"
    role             = "${aws_iam_role.lambda_process.arn}"
    handler          = "${var.lambda_function_name}.handler"
    runtime          = "nodejs8.10"
    memory_size      = "128"
    timeout          = "60"
    source_code_hash = "${base64sha256(file("lambda_process.zip"))}"
}

Other related discussion includes: this question on code hashing, (see my answer) and this GitHub issue.

SomeGuyOnAComputer
  • 5,414
  • 6
  • 40
  • 72
Kevin Buchs
  • 2,520
  • 4
  • 36
  • 55
  • Run the local-exec provisioner at the beginning of the aws_lambda_function resource one after another - they will run in sequence. – victor m Jan 10 '19 at 16:38
  • Why can't you zip up the entire directory contents? Why does your `source_code_hash` not use `var.lambda_zip` like your `command` does in your second `null_resource`? It would also help if you could show us the `tree` output of your directory. – SomeGuyOnAComputer Jan 11 '19 at 00:28
  • did you try the new feature in lambda, layers? so maintenant some layers with the proper version of node_modules packages, then you can directly deploy your nodejs codes easily – BMW Jan 11 '19 at 00:40
  • @victor m - no, they don't run in sequence. Terraform doesn't process sequentially. – Kevin Buchs Jan 11 '19 at 16:27
  • @SomeGuyOnAComputer, Yes, that would be a way to work around what I did. I have other files there, like terraform files. I could move the code to a subdirectory or make a copy when I run TF. However, I just wanted to stick with what I have. My real question, the title of my question, is not how can I solve my problem another way, but why didn't my original code function with proper ordering? – Kevin Buchs Jan 11 '19 at 16:27
  • @BMW - thanks for that idea. As I commented with SomeGuyOnAComputer, my question is really why my code does not work as expected. – Kevin Buchs Jan 11 '19 at 16:32
  • @KevinBuchs Multiple provisioners can be specified within a resource. Multiple provisioners are executed in the order they're defined in the configuration file see https://www.terraform.io/docs/provisioners/index.html – victor m Jan 11 '19 at 20:23
  • To answer your question - I think your code fails because of the null -resource trigger. If the lambda function files don't change, the null-resource triggers will not fire and lambda resource will fail. – victor m Jan 11 '19 at 20:30
  • @victorm Thanks. I now understand your point about multiple provisioners and sequencing. This was something new for me. Also, you mention that if the code doesn't change no zip file will be generated. Now, I guess there is the assumption that the file will always be there and it is just a matter of determining when it needs to be updated. If the file exists, then the lambda resource always works fine. This is as expected. ... continued – Kevin Buchs Jan 15 '19 at 23:24
  • The problem here is that the behavior was: change the source, tf apply and the zip file was not updated, but the existing zip file is uploaded to lambda. That file then had not changed since the last time. If I change nothing further but run tf apply again, then the zip file is updated, however, that updated zip file is not uploaded to lambda. If I change nothing but run tf apply again, then the updated zip file is uploaded to lambda. So, it takes 3 passes to get things right. – Kevin Buchs Jan 15 '19 at 23:25
  • try this: resource "null_resource" "make_zip" { provisioner "local-exec" { command = "zip -r ${var.lambda_zip} ${var.lambda_function_name}.js node_modules" } triggers = { source_hash = "${base64sha256(file(var.lambda_zip))}" } } – victor m Jan 16 '19 at 05:05
  • @victorm It would be better to use the [`archive_file`](https://www.terraform.io/docs/providers/archive/d/archive_file.html) data source to zip the contents on the fly. It will also not trigger an update if the code hasn't changed. – SomeGuyOnAComputer Jan 20 '19 at 18:35
  • @SomeGuyOnAComputer - I explain why this alternative doesn't work in my original question. And, my question is not seeking an alternative, but asking why what I presented as a solution was not working properly. – Kevin Buchs Jan 21 '19 at 21:06

0 Answers0