export string with substring from file with shell(bash)-script

Question

I have file variables.tf:

variable "do_token" {
  description = "set DO token value"
  type        = string
}

variable "ssh_pub_key_fingerprint" {
  description = "Set ssh key fingerprint stored in DO"
  type        = string
}

...

and i want to write script for export variables names with desctiptions like comment, to file terraform.tfvars. But first line must be #description, and second line variable with empty value and double quotes, like this:

cat terraform.tfvars

#set DO token value
do_token = ""

#Set ssh key fingerprint stored in DO
ssh_pub_key_fingerprint = ""

I tryed write bash script test.sh like this:

#!/bin/bash
echo -n "" > test.txt
grep 'description\|variable' variables.tf | while read line; do 
    OUTPUT=$(echo $line | sed 's/ =//g; s/ {//g' );
    # echo $OUTPUT
case "$OUTPUT" in 

  *description*)
    DESCRIPTION=$(echo $OUTPUT | awk -F "\"" '{print $2}')
    echo "#"$DESCRIPTION >> terraform.tfvars
    ;;

  *variable*)
    VARIABLE=$(echo $OUTPUT | awk -F "\"" '{print $2}')
    echo $VARIABLE " = \"\"">> terraform.tfvars
    ;;
esac    
done

but when i show file terraform.tfvars values line is a 1st, and description line 2nd but must be conversely

do_token  = ""
#set DO token value 

ssh_pub_key_fingerprint  = ""
#Set ssh key fingerprint stored in DO

how i can do this properly? Thanks

Note that if you're writing to a file more than once, it's inefficient to put `>>file` after each line that does an individual write, as that opens the file for just that one line, does the write, then closes it. Better to open it earlier in your script _and keep it open for reuse_. — Charles Duffy, Nov 21 '21 at 17:22
For example, `exec 3>>file` once will open the file then let you `echo "something" >&3` to write to that already-open file. That also lets you `exec 3>file` to clear the file's contents _only once_, after which each `echo "something" >&3` will just append to the already-open file (since truncation only happens on `open(..., O_TRUNC)`). — Charles Duffy, Nov 21 '21 at 17:23
Also, piping to a `while read` loop destroys all the variables set in the loop when the loop ends. See [BashFAQ #24](https://mywiki.wooledge.org/BashFAQ/024); it's better to do `while IFS= read -r line; do ...; done < <(grep ...)` than `grep ... | while IFS= read -r line`. — Charles Duffy, Nov 21 '21 at 17:24
Also, you should be quoting your expansions. `echo "$VARIABLE"`, not `echo $VARIABLE`. See [I just assigned a variable, but `echo $variable` shows something else!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) — Charles Duffy, Nov 21 '21 at 17:26
Also, all-caps names are reserved for variables that change the behavior of the shell itself; variables you define for your code's own use should use lower-case names. This is described in the POSIX standard at https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html -- read it keeping in mind that setting a regular shell variable overwrites any like-named environment variable. — Charles Duffy, Nov 21 '21 at 17:28

Charles Duffy · Accepted Answer · 2021-11-21T17:42:41.177

1

This whole program is better implemented in awk itself. Given your input,

#!/bin/bash
gawk '
  BEGIN { first = 1 }
  /variable/ {
    curr_var = gensub(/"/, "", "g", $2)
  }
  /description = ".*"/ {
    if (first != 1) { printf("\n") }
    printf("%s = \"\"\n", curr_var)
    printf("#%s\n", gensub(/.*["]([^"]+)["].*/, "\\1", $0))
    first=0
  }
'

...emits as output:

do_token = ""
#set DO token value

ssh_pub_key_fingerprint = ""
#Set ssh key fingerprint stored in DO

See this in the online sandbox at https://ideone.com/S8Fmz1

edited Nov 21 '21 at 17:42

answered Nov 21 '21 at 17:37

Charles Duffy

280,126
43
390
441

thanks, but in your sandbox example, i see {error} like: gawk: cmd. line:9: (FILENAME=- FNR=2) warning: gensub: third argument ` description = "set DO token value"' treated as 1 gawk: cmd. line:9: (FILENAME=- FNR=7) warning: gensub: third argument ` description = "Set ssh key fingerprint stored in DO"' treated as 1. how i can resolve it? – mocart Nov 21 '21 at 19:28
added third parameter - "g" to gensub printf("#%s\n", gensub(/.*["]([^"]+)["].*/, "\\1","g", $0)) and it works, thanks! – mocart Nov 21 '21 at 19:45

score 0 · Answer 2 · answered Nov 21 '21 at 17:42

0

If sed is an option

$ sed -n '/do_token/ {N;s/variable "\([^"]*\).*\n  description = "\([^"]*\).*/#\2\n\1 = ""\n/p};/ssh/{N;s/variable "\([^"]*\).*\n  description = "\([^"]*\).*/#\2\n\1 = ""/p}' input_file > terraform.tfvars

$ cat terraform.tfvars
#set DO token value
do_token = ""

#Set ssh key fingerprint stored in DO
ssh_pub_key_fingerprint = ""

answered Nov 21 '21 at 17:42

HatLess

10,622
5
14
32

it works, but only known (hardcoded in this command) variables, but if i add any other variable? i must insert all var name it in sed command? – mocart Nov 21 '21 at 19:23

export string with substring from file with shell(bash)-script

2 Answers2