What's a concise way to check that environment variables are set in a Unix shell script?

Question

I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:

if [ -z "$STATE" ]; then
    echo "Need to set STATE"
    exit 1
fi  

if [ -z "$DEST" ]; then
    echo "Need to set DEST"
    exit 1
fi

which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?

EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.

Answer 1

Parameter Expansion

The obvious answer is to use one of the special forms of parameter expansion:

: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}

Or, better (see section on 'Position of double quotes' below):

: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"

The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.

The second variant (using :?) requires DEST to be set and non-empty.

If you supply no message, the shell provides a default message.

The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.

It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:

${parameter:?word}

Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.

The Colon Command

I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.

---

Position of double quotes

blong asked in a comment:

Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749

The gist of the discussion is:

… However, when I shellcheck it (with version 0.4.1), I get this message:

In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
  ^-- SC2086: Double quote to prevent globbing and word splitting.

Any advice on what I should do in this case?

The short answer is "do as shellcheck suggests":

: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"

To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.

$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$ 
$ x="*"
$ printf "%s\n" ${x:?You must set x}    # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x}    # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}"  # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}"  # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"}  # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"}  # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"}  # Not quite careful enough
bash: x: You must set x
$ 

Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.

Answer 2

Try this:

[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;

Answer 3

Your question is dependent on the shell that you are using.

Bourne shell leaves very little in the way of what you're after.

BUT...

It does work, just about everywhere.

Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:

There are two possibilities here. The example above, namely using:

${MyVariable:=SomeDefault}

for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.

You also have the possibility of:

${MyVariable:-SomeDefault}

which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.

Answer 4

Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:

#!/bin/sh -u

This will cause the script to exit if any unbound variables lurk within.

Answer 5

${MyVariable:=SomeDefault}

If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.

The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:

MyVariable=${MyVariable:=SomeDefault}

Answer 6

In my opinion the simplest and most compatible check for #!/bin/sh is:

if [ "$MYVAR" = "" ]
then
   echo "Does not exist"
else
   echo "Exists"
fi

Again, this is for /bin/sh and is compatible also on old Solaris systems.

Answer 7

bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.

$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set

Answer 8

I always used:

if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi

Not that much more concise, I'm afraid.

Under CSH you have $?STATE.

Answer 9

For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:

#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)

for var_name in "${vars[@]}"
do
  if [ -z "$(eval "echo \$$var_name")" ]; then
    echo "Missing environment variable $var_name"
    exit 1
  fi
done

Answer 10

We can write a nice assertion to check a bunch of variables all at once:

#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
  local fatal var num_null=0
  [[ "$1" = "-f" ]] && { shift; fatal=1; }
  for var in "$@"; do
    [[ -z "${!var}" ]] &&
      printf '%s\n' "Variable '$var' not set" >&2 &&
      ((num_null++))
  done

  if ((num_null > 0)); then
    [[ "$fatal" ]] && exit 1
    return 1
  fi
  return 0
}

Sample invocation:

one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute

Output:

test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set

---

More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh

Answer 11

None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:

mapfile -t arr < variables.txt

EXITCODE=0

for i in "${arr[@]}"
do
   ISSET=$(env | grep ^${i}= | wc -l)
   if [ "${ISSET}" = "0" ];
   then
      EXITCODE=-1
      echo "ENV variable $i is required."
   fi
done

exit ${EXITCODE}

Answer 12

This can be a way too:

if (set -u; : $HOME) 2> /dev/null
...
...

http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html

Answer 13

Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:

is_this_an_env_variable ()
    local var="$1"
    if env |grep -q "^$var"; then
       return 0
    else
       return 1
    fi
 }

Answer 14

The $? syntax is pretty neat:

if [ $?BLAH == 1 ]; then 
    echo "Exists"; 
else 
    echo "Does not exist"; 
fi