14

I am using the Terraform archive_file provider to package multiple files into a zip file. It works fine when I define the archive like this:

data "archive_file" "archive" {
  type        = "zip"
  output_path = "./${var.name}.zip"
  source_dir  = "${var.source_dir}"
}

However I don't want the archive to contain all of the files in var.source_dir, I only want a subset of them. I notice the archive_file provider has a source_file attribute so I was hoping I could supply a list of those files and package them into the archive like so:

locals {
  source_files = ["${var.source_dir}/foo.txt", "${var.source_dir}/bar.txt"]
}

data "archive_file" "archive" {
  type        = "zip"
  output_path = "./${var.name}.zip"
  count       = "2"
  source_file = "${local.source_files[count.index]}"
}

but that doesn't work, the archive gets built for each file defined in local.source-files hence I have a "last one wins" scenario where the archive file that gets built only contains bar.txt.

I tried this:

locals {
  source_files = ["${var.source_dir}/main.py", "${var.source_dir}/requirements.txt"]
}

data "archive_file" "archive" {
  type        = "zip"
  output_path = "./${var.name}.zip"
  source_file = "${local.source_files}"
}

but unsurprisingly that failed with:

data.archive_file.archive: source_file must be a single value, not a list

Is there a way to achieve what I'm after here i.e pass a list of files to the archive_file provider and have it package all of them into the archive file?

jamiet
  • 10,501
  • 14
  • 80
  • 159

2 Answers2

8

---- Thanks jamiet, I modified as your comment ----

  1. copy files to temp dir and archive them
locals {
  source_files = ["${var.source_dir}/main.py", "${var.source_dir}/requirements.txt"]
}

data "template_file" "t_file" {
  count = "${length(local.source_files)}"

  template = "${file(element(local.source_files, count.index))}"
}

resource "local_file" "to_temp_dir" {
  count    = "${length(local.source_files)}"
  filename = "${path.module}/temp/${basename(element(local.source_files, count.index))}"
  content  = "${element(data.template_file.t_file.*.rendered, count.index)}"
}

data "archive_file" "archive" {
  type        = "zip"
  output_path = "${path.module}/${var.name}.zip"
  source_dir  = "${path.module}/temp"

  depends_on = [
    "local_file.to_temp_dir",
  ]
}
  1. use source of archive_file
locals {
  source_files = ["${var.source_dir}/main.py", "${var.source_dir}/requirements.txt"]
}

data "template_file" "t_file" {
  count = "${length(local.source_files)}"

  template = "${file(element(local.source_files, count.index))}"
}


data "archive_file" "archive" {
  type        = "zip"
  output_path = "./${var.name}.zip"

  source {
    filename = "${basename(local.source_files[0])}"
    content  = "${data.template_file.t_file.0.rendered}"
  }

  source {
    filename = "${basename(local.source_files[1])}"
    content  = "${data.template_file.t_file.1.rendered}"
  }
}
  1. create shell script and call it using external data resource.
locals {
  source_files = ["${var.source_dir}/main.py", "${var.source_dir}/requirements.txt"]
}

data "template_file" "zip_sh" {
  template = <<EOF
#!/bin/bash
zip $* %1>/dev/null %2>/dev/null
echo '{"result":"success"}'
EOF
}

resource "local_file" "zip_sh" {
  filename = "${path.module}/zip.sh"
  content  = "${data.template_file.zip_sh.rendered}"
}

data "external" "zip_sh" {
  program = ["${local_file.zip_sh.filename}", "${var.name}", "${join(" ", local.source_files)}"]

  depends_on = [
    "data.template_file.zip_sh",
  ]
}
RyanKim
  • 1,557
  • 9
  • 10
  • I went with option1 and it worked great. Thank you. – jamiet Jul 08 '19 at 08:22
  • Ah, one small modification. `template = "${element(local.source_files, count.index)}"` needed to be `template = "${file(element(local.source_files, count.index))}"` – jamiet Jul 08 '19 at 14:59
  • have hit another snag with this as I modify the solution to allow a source_root_dir to be defined and the file locations are then relative to source_root_dir (want to do this so I can copy with files in subdirectories), posted at https://stackoverflow.com/questions/56942962/terraform-nested-interpolation-with-count – jamiet Jul 08 '19 at 22:03
2

Well theres another compact way to do is using excludes parameter

data "archive_file" "archive" {
  type        = "zip"
  output_path = "${path.module}/${var.name}.zip"
  source_dir  = "${path.module}/${var.source_dir}/"
  excludes    = setsubtract(fileset("${var.source_dir}/","*"), ["requirements.txt", "main.py"])
}

This will help you to include selective files/folders/sub-folders but your state file might look a little ugly.

Ripe
  • 21
  • 2