Reading key/value parameters from a file into a shell script

Question

I've got my script working almost. The goal of this is to take defined values from a file and then populate these valeus in a shell script at runtime.

Please take a look at what i have here...

The first file: ab.sh

#!/bin/bash
USER_TYPE=$1 #IDENTIFY USER TYPE TYPE1,TYPE2,TYPE3,TYPE4
USERNAME=$2
PERMISSION_TYPE=$3 #IDENTIFY PERMISSION TYPE SELECT,UPDATE,DELETE,INSERT
TARGET_USER=$4
TARGET_TABLE=$5

if [ $USER_TYPE == 'TYPE1' ]
        then
cat Parameters.conf |while read USERNAME
do
sqlplus / as sysdba <<  E00
CREATE USER ${USERNAME}
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
ACCOUNT UNLOCK;
ALTER USER ${USERNAME} DEFAULT ROLE ALL;
GRANT CREATE SESSION TO ${USERNAME};
GRANT CONNECT TO ${USERNAME};
exit
E00
done

cat p.conf |while read PERMISSION_TYPE TARGET_SCHEMA TARGET_TABLE USERNAME
do
sqlplus / as sysdba > /home/o/output/output.log << E01
GRANT ${PERMISSION_TYPE} ON ${TARGET_USER}.${TARGET_TABLE} TO ${USERNAME};
E01
done
fi

This is the file Parameters.conf where the values are defined and should come from...

Parameters.conf

USER_TYPE TYPE1
USERNAME NEWUSER
PERMISSION_TYPE SELECT,UPDATE
TARGET_USER TESTUSER
TARGET_TABLE ABC

Don't use all-caps names for your own variables -- this makes them conflict with environment variables defined by the operating system and the shell, which are restricted to that namespace (for instance, there's a `USER` preset in bash which you've overriding here). See http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html, fourth paragraph, for POSIX conventions around environment variable names (and by extension shell variable names, since they share a namespace). — Charles Duffy, Aug 16 '16 at 00:36
Sure; that said, the point was more about convention -- if you were using `user` or `username` as opposed to `USER` or `USERNAME`, you wouldn't need to be policing which specific variables your OS or shell is or isn't setting and whether there's a conflict. — Charles Duffy, Aug 16 '16 at 00:40
...btw, which version of bash is this going to be run with? As of 4.0, there's associative array functionality available, so we can keep variables read from the file in a completely different namespace from shell variables. That same effect could be done with, say, prefixing or suffixing if compatibility with other versions is needed. — Charles Duffy, Aug 16 '16 at 00:51

Charles Duffy · Accepted Answer · 2016-08-16T00:58:19.733

First choice: Associative array

This requires bash 4.0 or newer to support declare -A; see below for other options.

#!/bin/bash

# first, read your key/value pairs into shell variables
declare -A v=( )
while read -r var value; do
  v[$var]=$value
done < Parameters.conf

# second, perform a command that depends on them
sqlplus / as sysdba <<  E00
CREATE USER ${v[USERNAME]}
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
ACCOUNT UNLOCK;
ALTER USER ${v[USERNAME]} DEFAULT ROLE ALL;
GRANT CREATE SESSION TO ${v[USERNAME]};
GRANT CONNECT TO ${v[USERNAME]};
exit
E00

sqlplus / as sysdba > /home/o/output/output.log << E01
GRANT ${v[PERMISSION_TYPE]} ON ${v[TARGET_USER]}.${v[TARGET_TABLE]} TO ${v[USERNAME]};
E01

Key points:

Assignment to a shell variable which is itself named in a variable is described in BashFAQ #6; likewise for associative arrays.
The redirection needs to be done as while read key value; do ...; done <input rather than cat input | while read key value; do ...; done to avoid the bug BashFAQ #24.
The actual sqlplus calls shouldn't be inside the loop, because the loop body is run once per line in the file. Since you want all the file's lines to be read (and all the variables assigned) before running sqlplus, the loop should entirely complete before sqlplus is called.
Using either a prefix or an associative-array ensures that variables coming from the configuration file can't override system environment variables like PATH or LD_PRELOAD.

Second choice: Prefixed namespace

#!/bin/bash

while read -r var value; do
  printf -v "v_$var" %s "$value"
done <Parameters.conf

# second, perform a command that depends on them
sqlplus / as sysdba <<  E00
CREATE USER ${v_USERNAME}
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
ACCOUNT UNLOCK;
ALTER USER ${v_USERNAME} DEFAULT ROLE ALL;
GRANT CREATE SESSION TO ${v_USERNAME};
GRANT CONNECT TO ${v_USERNAME};
exit
E00

sqlplus / as sysdba > /home/o/output/output.log << E01
GRANT ${v_PERMISSION_TYPE} ON ${v_TARGET_USER}.${v_TARGET_TABLE} TO ${v_USERNAME};
E01

so it'll be: while read Parameter1 parameter2 so-on and so forth — Tony Sawah, Aug 16 '16 at 00:42
Quite correct. You might make it `while read -r parameter1 parameter2 ...` to avoid `read`'s default behavior of parsing (and thus deleting) backslashes in input. (Even if you don't *expect* backslashes, it typically makes sense to turn off that behavior unless you explicitly have a reason to want it). — Charles Duffy, Aug 16 '16 at 00:43
Thank you so much. I'm so appreciative for this. I understand it now completely! — Tony Sawah, Aug 16 '16 at 00:54
Glad to help! And do accept my apologies for not paying enough attention earlier. (I've also amended to try to show two different approaches to separating configuration variables from others -- associative arrays, which other languages might call "hashes" or "maps"; and using a prefixed namespace). — Charles Duffy, Aug 16 '16 at 00:59

Reading key/value parameters from a file into a shell script

1 Answers1

First choice: Associative array

Second choice: Prefixed namespace

Linked

Related