0

I have following loop, script, bash 3.2 is installed:

#!/bin/bash -e

set -e
set -u
set -o pipefail

somefunc() {

    jq -c '.[]' <"${imports_json}" | while read -r i; do
      address=$(echo "$i" | jq -r '.address')
      item_id=$(echo "$i" | jq -r '.id')
      echo "Importing [${address}], Id: [${item_id}]"
      terraform -chdir="${terraform_dir}" import -state="${state}" "${address}" "${item_id}"
    done

    echo "Import loop is done ..."
}

...

# First invoke, all iterated
somefunc()

some other script code

# Second invoke, single import while expected to go over long list
somefunc()

It is simple loop over json array:

[
  {
    "address": "some_terraform_resource.resource[\"some_name\"]",
    "id": "some_resource_id"
  },
  {
    "address": "another_terraform_resource.resorce",
    "id": "another_id"
  }
...
]

I run it in several places in long script

In one place it is just doing one iteration and quits the loop, with no error code and echo "Import loop is done ..." following the loop executed. So, as if this terraform import command acts as a break.

If I comment terraform import command out all array is iterated. If I change json content, is just one single import and break, no matter what resource is imported.

Just for experiment, I've tried:

  • to suppress output (> /dev/null, 2>&1).
  • || true, even though terraform import is returning 0 error code.
  • comment out set commands at the beginning of the script
  • run terraform import ... in subshell, (terraform import ....)

No effect in all cases

Trace with bash -x:

Importing [some_terraform_resource.resource["some_name"]], Id: [some_resource_id], state: "/Users/<some-path>/baseline.tfstate"
+ terraform -chdir=/Users/<some-path>/project-for-import import -state=/Users/<some-path>/baseline.tfstate 'some_terraform_resource.resource["some_name"]' some_resource_id
+ read -r i
+ echo 'Terraform import is done'

I am pulling my hair out, what can cause loop to break ? What are the ways to troubleshoot it?

Vetal
  • 275
  • 1
  • 3
  • 13
  • What about if you drop `set -e`? – tshiono Apr 28 '23 at 01:03
  • 1
    I am not terribly familiar with `terraform`, but it seems like importing the same resources with the same ids multiple times might be an problematic. – j_b Apr 28 '23 at 01:08
  • 4
    Does terraform read from standard input? If so, it may be stealing the rest of the list. See [BashFAQ #89](https://mywiki.wooledge.org/BashFAQ/089). – Gordon Davisson Apr 28 '23 at 01:26
  • 1
    Did you try `terraform ... < /dev/null`? – Renaud Pacalet Apr 28 '23 at 04:44
  • 2
    Unrelated, but running 3 copies of `jq` seems inefficient (and potentially error-prone). Why not `jq -r '.[] | " \(.id) \(.address)"' <"${imports_json}" | while read -r id address; do …; done` – knittl Apr 28 '23 at 07:52
  • `while` with `read` will read lines. Are you certain your input is exactly the same on all systems? It could be that the system where it fails had wrong line termination characters (check Windows - Linux end of lines). – Nic3500 Apr 28 '23 at 08:38

1 Answers1

0

Indeed, terraform "stolen" STDIN, with the chunk fed from read, matching input variable input. Which I haven't noticed, while followed by a follow-up script output.

Gordon Davisson and Renaud Pacalet replies nailed the issue.

Adding < /dev/null makes terraform terminate the script with a clearly seen unsatisfied variable prompt

I'll apply other suggestions as well to optimize the script. Thanks a lot for your valuable input!

Vetal
  • 275
  • 1
  • 3
  • 13
  • 1
    If terraform needs to be able to prompt for user input from within the loop, you can read the JSON over a different file descriptor with e.g. `while read -r i <&3; do ... done 3< <(jq -c '.[]' <"${imports_json}")` – Gordon Davisson Apr 30 '23 at 07:48