Bash - Expand variable references in variables read via while read loop

Question

Bash version : 5.0.3(1)-ui-release

I don't know where I'm wrong.

1. One file "install_options.sh" with variables inside, like :

ADMIN_EMAIL="contact@john-doe.fr"                               
ADMIN_FIRST_NAME="John"                                                  
ADMIN_NAME="Doe" 
....

2. another file : "admin_settings.txt" with data like "KEY VALUE" (withespace separator), and few value is variable:

KEY VALUE
KEY "${ADMIN_EMAIL}"
NEED_HELP PLEASE
KEY "${ADMIN_FIRST_NAME}"
...

3. When i try on cli:

$ echo "${ADMIN_EMAIL}"

I have the right output:

contact@john-doe.fr

4. I try to use while read loop:

#!/bin/bash

while read IFS= -r KEY VALUE; do
 
COMMAND $KEY "${VALUE}"
 
done < admin_settings.txt

I get output:

KEY "${ADMIN_EMAIL}"

I would like get the output with the value:

KEY contact@john-doe.fr

I have tried many things, like put source file inside the while read loop, but not working:

#!/bin/bash

while read IFS= -r KEY VALUE; 
do
   source install_options.sh
   COMMAND $KEY "${VALUE}"

done < admin_settings.txt

Inside "admin_settings.txt", I have tried with:

KEY "${ADMIN_EMAIL}"
KEY ${ADMIN_EMAIL}
KEY "$ADMIN_EMAIL"
KEY $ADMIN_EMAIL

Same with COMMAND inside while loop, I have tried:

COMMAND $KEY "${VALUE}" 
COMMAND $KEY ${VALUE} 
COMMAND $KEY "$VALUE"
COMMAND $KEY $VALUE

What can I do to get the expected output?

Tanks for your repply, inside the "admin_settings.txt", I have try : KEY "${ADMIN_EMAIL}" KEY ${ADMIN_EMAIL} KEY "$ADMIN_EMAIL" KEY $ADMIN_EMAIL And try to leave the shebang, but i dont get the excpected output. — Mig, Sep 02 '22 at 14:33
So you can modify `admin_settings.txt` as you want? using `KEY $ADMIN_EMAIL` would be the easiest solution because it'll allows to use `envsubst` — Fravadona, Sep 02 '22 at 15:52
It would be impossible to write secure code handling untrusted data in bash if variable references, command substitutions, and other syntax were silently expanded _when present in data_ as this question seems to assume will happen! — Charles Duffy, Sep 02 '22 at 16:37
I'm pretty sure you don't want to be doing `source install_options.sh` in the loop body, for every iteration. — Kaz, Sep 02 '22 at 17:23
Please [edit] your question to show the expected output given your posted sample input. — Ed Morton, Sep 02 '22 at 17:54

Fravadona · Answer 1 · 2022-09-02T17:19:35.197

Assuming that admin_settings.txt is EXACTLY in the following format (no shebang, no blank lines, no comments, no double-quotes around variables, etc...):

KEY0 VALUE0
KEY1 $ADMIN_EMAIL
NEED_HELP PLEASE
KEY2 $ADMIN_FIRST_NAME

Then you can use envsubst for substituting the variables with their value:

set -a
source install_options.sh
set +a

envsubst < admin_settings.txt |
while IFS=' ' read -r key val
do
    COMMAND "$key" "$val"
done

Mig · Answer 2 · 2022-09-02T18:13:38.077

0

In this setup i cant use envsubst , but i just found another way with for loop and arrays, like :

source install_options.sh

VALUE=("$ADMIN_EMAIL" "$ADMIN_FIRST_NAME" "$ADMIN_NAME") 
KEY=("KEY0" "KEY1" "KEY2_TANKS_FOR_YOUR_REPPLY")    

for (( z=0; z<${#VALUE[@]}; z++ )); do

    COMMAND ${KEY[$z]} ${VALUE[$z]}

done

edited Sep 02 '22 at 18:13

answered Sep 02 '22 at 17:51

Mig

1
1

It's not clear how that code is related to your question. For example, where are your input files `install_options.sh` and `admin_settings.txt`? Also, copy/paste that into http://shellcheck.net and fix the issues it tells you about and read [correct-bash-and-shell-script-variable-capitalization](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization) then fix those issues too. Your loop is also more complicated than it needs to be and if you can have arrays as your starting point you could just use 1 associative array instead of 2 indexed arrays. – Ed Morton Sep 02 '22 at 17:57
I forgot to mention 'install_options.sh', I just edited my answer, and all data inside 'admin_settings.txt' has been transferred into the arrays. Thanks for your advices, tools, and links, I'll read carefully ! – Mig Sep 02 '22 at 18:29
You're welcome but the main point is that this isn't an answer to the question you asked about how to read and then use values from a file (`admin_settings.txt`) that has a mix of key-value pairs and other text, it's a possible answer (but don't do it as it's buggy) to a much simpler problem. – Ed Morton Sep 02 '22 at 18:31

Bash - Expand variable references in variables read via while read loop

2 Answers2