Parsing variables from config file in Bash

Question

Having the following content in a file:

VARIABLE1="Value1"
VARIABLE2="Value2"
VARIABLE3="Value3"

I need a script that outputs the following:

Content of VARIABLE1 is Value1
Content of VARIABLE2 is Value2
Content of VARIABLE3 is Value3

Any ideas?

score 67 · Answer 1 · answered May 16 '13 at 00:30

67

Since your config file is a valid shell script, you can source it into your current shell:

. config_file
echo "Content of VARIABLE1 is $VARIABLE1"
echo "Content of VARIABLE2 is $VARIABLE2"
echo "Content of VARIABLE3 is $VARIABLE3"

Slightly DRYer, but trickier

. config_file
for var in VARIABLE1 VARIABLE2 VARIABLE3; do
    echo "Content of $var is ${!var}"
done

answered May 16 '13 at 00:30

glenn jackman

238,783
38
220
352

12

@samson for one thing, it allows any and all malicious code to be injected. – Big McLargeHuge May 28 '15 at 03:35
8

Why would you worry about malicious code being injected from a config file? I suppose I could imagine such cases, but I needed to be able to do this from a file which only exists on my local machine, i.e. a list of user / password combinations... – samson May 28 '15 at 21:02
You need to hard-code variable names in this answer. With the accepted answer, you do not, and that's what makes it much better. – kara deniz Mar 08 '16 at 20:13
1

@samson, I'd worry about that because the way small security breaches become large security breaches is attackers finding exploits they can leverage to increase their (often-initially-minimal) access. A configuration file with carelessly-assigned permissions is often just such an opportunity. – Charles Duffy Sep 16 '16 at 12:56
I would also prefer the accepted solution here because it works in more general cases and does it automatically independent of the variable names and amount – derHugo Jun 21 '17 at 10:00
There's also a risk if a value is omitted from the config file. If a user is allowed to run the script but not modify the configuration, they may be able to circumvent the restriction by setting environment variables. You would need to protect the environment as an attack vector. – Drew Nutter Mar 16 '22 at 22:08

abasu · Accepted Answer · 2013-05-16T14:30:46.403

47

awk -F\= '{gsub(/"/,"",$2);print "Content of " $1 " is " $2}' <filename>

Just FYI, another pure bash solution

IFS="="
while read -r name value
do
echo "Content of $name is ${value//\"/}"
done < filename

edited May 16 '13 at 14:30

answered May 15 '13 at 17:56

abasu

2,454
19
22

You can also use `IFS="=" read -r name value` to handle the splitting (the `-r` has nothing to do with splitting, but is recommended unless you have a reason not to use it). – chepner May 16 '13 at 14:08
The awk and bash solutions will give different results if the value contains an equals sign. – glenn jackman Aug 08 '23 at 21:22

score 40 · Answer 3 · edited Apr 16 '18 at 16:32

If you need these...

Features

Single line and inline comments;
Trimming spaces around = (ie var = value will not fail);
Quoted string values;
Understanding of DOS line endings;
Keep safe, avoiding sourcing your config file.

Code

shopt -s extglob
configfile="dos_or_unix" # set the actual path name of your (DOS or Unix) config file
tr -d '\r' < $configfile > $configfile.unix
while IFS='= ' read -r lhs rhs
do
    if [[ ! $lhs =~ ^\ *# && -n $lhs ]]; then
        rhs="${rhs%%\#*}"    # Del in line right comments
        rhs="${rhs%%*( )}"   # Del trailing spaces
        rhs="${rhs%\"*}"     # Del opening string quotes 
        rhs="${rhs#\"*}"     # Del closing string quotes 
        declare $lhs="$rhs"
    fi
done < $configfile.unix

Comments

tr -d '\r' ... deletes DOS carriage return.
! $lhs =~ ^\ *# skips single line comments and -n $lhs skips empty lines.
Deleting trailing spaces with ${rhs%%*( )} requires setting extended globbing with shopt -s extglob. (Apart using sed), you can avoid this, via the more complex:

rhs="${rhs%"${rhs##*[^ ]}"}"

Test config file

## This is a comment 
var1=value1             # Right side comment 
var2 = value2           # Assignment with spaces 

## You can use blank lines 
var3= Unquoted String   # Outer spaces trimmed
var4= "My name is "     # Quote to avoid trimming 
var5= "\"Bob\""

Test code

echo "Content of var1 is $var1"
echo "Content of var2 is $var2"
echo "Content of var3 is [$var3]"
echo "Content of var4 + var5 is: [$var4$var5]"

Results

Content of var1 is value1
Content of var2 is value2
Content of var3 is [Unquoted String]
Content of var4 + var5 is: [My name is "Bob"]

Isn't rhs="${rhs%\"*}" for the closing quotes and rhs="${rhs#\"*}" for the opening quotes? — meso_2600, Feb 03 '16 at 14:58
and another issue: when the value is "aaaa#AAAAA" script strips everything on the rhs from # — meso_2600, Feb 12 '16 at 10:37
I can confirm this script fails if there's a `#` in the value. — Arda, May 13 '16 at 09:13

score 9 · Answer 4 · answered Oct 27 '15 at 19:29

9

I do in this way

. $PATH_TO_FILE

answered Oct 27 '15 at 19:29

albert

1,766
1
21
24

2

similar to what Dave Kennedy said, this allows any and all malicious code to be injected – michaelmcandrew Feb 19 '16 at 23:15
5

Only if someone change the original file in "$PATH_TO_FILE" on the other. If someone can do this, so, the system admin are sloppy. – albert Feb 21 '16 at 00:18

score 4 · Answer 5 · answered May 15 '13 at 17:56

4

awk '{print "Content of "$1" is "$3}' FS='[="]'

Result

Content of VARIABLE1 is Value1
Content of VARIABLE2 is Value2
Content of VARIABLE3 is Value3

answered May 15 '13 at 17:56

Zombo

1
62
391
407

score 0 · Answer 6 · answered Oct 17 '17 at 22:07

given a config file as follows :-

[a]
b=C
d=E;rm t1
[b]
g=h

the following one-liner will parse and hold the values :-

CFG=path-to-file; for ini in `awk '/^\[/' $CFG`;do unset ARHG;declare -A ARHG;while read A B;do ARHG[$A]=$B;echo "in section $ini, $A is equal to"  ${ARHG["$A"]};done < <(awk -F'=' '/\[/ {x=0} x==1 && $0~/=/ && NF==2 {print $1, $2} $0==INI {x=1}' INI="$ini" $CFG);declare -p ARHG;echo;done;printf "end loop\n\n";declare -p ARHG

Now, let's break that down

CFG=path-to-file;
for ini in `awk '/^\[/' $CFG` # finds the SECTIONS (aka "ini")
do 
  unset ARHG # resets ARHG 
  declare -A ARHG # declares an associative array
  while read A B
  do
    ARHG[$A]=$B
    echo "in section $ini, $A is equal to"  ${ARHG["$A"]}
  done < <(awk -F'=' '/\[/ {x=0} x==1 && $0~/=/ && NF==2 {print $1, $2} $0==INI {x=1}' INI="$ini" $CFG)
  # the awk splits the file into sections, 
  # and returns pairs of values separated by "="
  declare -p ARHG # displays the current contents of ARHG
  echo
done
printf "end loop\n\n"
declare -p ARHG

This allows us to save values, without using eval or backtick. To be "really clean", we could remove [:space:] at the start and end of line, ignore "^#" lines, and remove spaces around the "equals" sign.

of course, the use of "declare -A" requires a recent version of bash. For example, "GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14)" does NOT include the "-A" parameter ... — Rich Armstrong, Oct 17 '17 at 22:24

score -1 · Answer 7 · answered Jul 21 '17 at 11:13

    # 
    #------------------------------------------------------------------------------
    # parse the ini like $0.$host_name.cnf and set the variables
    # cleans the unneeded during after run-time stuff. Note the MainSection
    # courtesy of : http://mark.aufflick.com/blog/2007/11/08/parsing-ini-files-with-sed
    #------------------------------------------------------------------------------
    doParseConfFile(){
        # set a default cnfiguration file
        cnf_file="$run_unit_bash_dir/$run_unit.cnf"

        # however if there is a host dependant cnf file override it
        test -f "$run_unit_bash_dir/$run_unit.$host_name.cnf" \
            && cnf_file="$run_unit_bash_dir/$run_unit.$host_name.cnf"

        # yet finally override if passed as argument to this function
        # if the the ini file is not passed define the default host independant ini file
        test -z "$1" || cnf_file=$1;shift 1;


        test -z "$2" || ini_section=$2;shift 1;
        doLog "DEBUG read configuration file : $cnf_file"
        doLog "INFO read [$ini_section] section from config file"

        # debug echo "@doParseConfFile cnf_file:: $cnf_file"
        # coud be later on parametrized ...
        test -z "$ini_section" && ini_section='MAIN_SETTINGS'

        doLog "DEBUG reading: the following configuration file"
        doLog "DEBUG ""$cnf_file"
        ( set -o posix ; set ) | sort >"$tmp_dir/vars.before"

        eval `sed -e 's/[[:space:]]*\=[[:space:]]*/=/g' \
            -e 's/#.*$//' \
            -e 's/[[:space:]]*$//' \
            -e 's/^[[:space:]]*//' \
            -e "s/^\(.*\)=\([^\"']*\)$/\1=\"\2\"/" \
            < $cnf_file \
            | sed -n -e "/^\[$ini_section\]/,/^\s*\[/{/^[^#].*\=.*/p;}"`

        ( set -o posix ; set ) | sort >"$tmp_dir/vars.after"

        doLog "INFO added the following vars from section: [$ini_section]"
        cmd="$(comm -3 $tmp_dir/vars.before $tmp_dir/vars.after | perl -ne 's#\s+##g;print "\n $_ "' )"
        echo -e "$cmd"
        echo -e "$cmd" >> $log_file
        echo -e "\n\n"
        sleep 1; printf "\033[2J";printf "\033[0;0H" # and clear the screen
    }
    #eof func doParseConfFile