Search and replace variables in a file using bash/sed

Question

I am trying to write a bash script(script.sh) to search and replace some variables in input.sh file. But I need to modify only the variables which are present in variable_list file and leave others as it is.

variable_list

${user}  
${dbname}

input.sh

username=${user}  
password=${password}  
dbname=${dbname}

Expected output file

username=oracle  
password=${password} > This line won't be changed as this variable(${password}) is not in variable_list file
dbname=oracle

Following is the script I am trying to use but I am not able to find the correct sed expression

script.sh

export user=oracle  
export password=oracle123  
export dbname=oracle

variable='variable_list'  
while read line ;  
do  
 if [[ -n $line ]]     
 then  
  sed -i 's/$line/$line/g' input.sh > output.sh  
 fi  
done < "$variable"    

Answer 1

This could work:

#!/bin/bash

export user=oracle  
export password=oracle123  
export dbname=oracle

variable='variable_list'  
while read line ;  
do  
 if [[ -n $line ]]
 then
  exp=$(sed -e 's/\$/\\&/g' <<< "$line")
  var=$(sed -e 's/\${\([^}]\+\)}/\1/' <<< "$line")
  sed -i "s/$exp/${!var}/g" input.sh
 fi  
done < "$variable"

The first sed expression escapes the $ which is a regex metacharacter. The second extracts just the variable name, then we use indirection to get the value in our current shell and use it in the sed expression.

Edit

Rather than rewriting the file so many times, it's probably more efficient to do it like this, building the arguments list for sed:

#!/bin/bash

export user=oracle  
export password=oracle123  
export dbname=oracle

while read var
do
    exp=$(sed -e 's/\$/\\&/g' <<< "$var")
    var=$(sed -e 's/\${\([^}]\+\)}/\1/' <<< "$var")
    args+=("-e s/$exp/${!var}/g")
done < "variable_list"

sed "${args[@]}" input.sh > output.sh

Answer 2

Here is a script.sh that works:

#!/bin/bash

user=oracle
password=oracle123
dbname=oracle

variable='variable_list'
text=$(cat input.sh)
while read line
do
  value=$(eval echo $line)  
  text=$(sed "s/$line/$value/g" <<< "$text")
done < "$variable"
echo "$text" > output.sh

Note that your original version contains single quotes around the sed string, which doesn't insert the value of $line. It is trying to look for the literal line after the end of the line $ (which will never find anything).

Since you are looking for the value of the variable in $line, you need to do an eval to get this.

Also, since there are multiple variables you are looping over, the intermediate text variable stores the result as it loops.

The export keyword is also unnecessary in this script, unless it is being used in some sub-process not shown.

Answer 3

user=oracle  
password=oracle123  
dbname=oracle

variable_list=( '${user}' '${dbname}' )

while IFS="=$IFS" read variable value; do
    for subst_var in "${variable_list[@]}"; do
        if [[ $subst_var = $value ]]; then
            eval "value=$subst_var"
            break
        fi
    done
    printf "%s=%s\n" "$variable" "$value"
done < input.sh > output.sh

Answer 4

TXR solution. Build a filter dynamically. The filter is implemented internally as a trie data structure, which gives us a lex-like state machine which matches the entire dictionary at once as the input is scanned. For simplicity, we include the ${ and } as part of the variable name.

@(bind vars (("${user}" "oracle")
             ("${dbname}" "oracle")
             ("${password}" "letme1n")))
@(next "variable_list")
@(collect)
@entries
@(end)
@(deffilter subst . @(mapcar (op list @1 (second [find vars @1 equal first]))
                             entries))
@(next "input.sh")
@(collect)
@line
@  (output :filter subst)
@line
@  (end)
@(end)

Run:

$ txr subst.txr
username=oracle
password=${password}
dbname=oracle

input.sh: (as given)

username=${user}
password=${password}
dbname=${dbname}

variable_list: (as given)

${user}
${dbname}