How to source a script with environment variables in a docker build process?

Question

I'm creating an image that has a similar problem like the following docker project:

Dockerfile

FROM alpine:3.9.3

COPY ./env.sh /env.sh
RUN source /env.sh
CMD env

env.sh

TEST=test123

I built the image with

docker build -t sandbox .

and run it with

docker run --rm sandbox

The output is

HOSTNAME=72405c43801b
SHLVL=1
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/

My environment variable is missing.

In the real project I have to source a longer complex script for the installation for IBM DB2 client that also sets environment variables. How can I achieve it without reading the whole installation process and setting all variables with ENV in the dockerfile?

EDIT: In the real project the file env.sh is created as part of the installation process and it is not available from outside of the container. The environment variables are set depending on the system it is executed on. If I run it on the host it will set wrong variables in the guest.

Part of the real script is

if [ -f ${INST_DIR?}/tools/clpplus.jar ]; then
    AddRemoveString CLASSPATH ${INST_DIR?}/tools/clpplus.jar a
fi

if [ -f ${INST_DIR?}/tools/antlr-3.2.jar ]; then
    AddRemoveString CLASSPATH ${INST_DIR?}/tools/antlr-3.2.jar a
fi

if [ -f ${INST_DIR?}/tools/jline-0.9.93.jar ]; then
    AddRemoveString CLASSPATH ${INST_DIR?}/tools/jline-0.9.93.jar a
fi

if [ -f ${INST_DIR?}/java/db2jcc.jar ]; then
    AddRemoveString CLASSPATH ${INST_DIR?}/java/db2jcc.jar a
fi

if [ -f ${INST_DIR?}/java/db2jcc_license_cisuz.jar ]; then
    AddRemoveString CLASSPATH ${INST_DIR?}/java/db2jcc_license_cisuz.jar a
fi

It checks the installation and sets the variables depending on this. Since on the host is no DB2 installation the variables wouldn't be set.

Answer 1

Each Dockerfile RUN step runs a new container and a new shell. If you try to set an environment variable in one shell, it will not be visible later on. For example, you might experiment with this Dockerfile:

FROM busybox
ENV FOO=foo1
RUN export FOO=foo2
RUN export BAR=bar
CMD echo FOO is $FOO, BAR is $BAR
# Prints "FOO is foo1, BAR is "

There are three good solutions to this. In order from easiest/best to hardest/most complex:

1. Avoid needing the environment variables at all. Install software into “system” locations like /usr; it will be isolated inside the Docker image anyways. (Don’t use an additional isolation tool like Python virtual environments, or a version manager like nvm or rvm; just install the specific thing you need.)

2. Use ENV. This will work:

   FROM busybox
   ENV FOO=foo2
   ENV BAR=bar
   CMD echo FOO is $FOO, BAR is $BAR
   # Prints "FOO is foo2, BAR is bar"
   

3. Use an entrypoint script. This typically looks like:

   #!/bin/sh
   # Read in the file of environment settings
   . /opt/wherever/env
   # Then run the CMD
   exec "$@"
   

   COPY this script into your Dockerfile. Make it be the ENTRYPOINT; make the CMD be the thing you’re actually running.

   FROM busybox
   WORKDIR /app
   COPY entrypoint.sh .
   COPY more_stuff .
   ENTRYPOINT ["/app/entrypoint.sh"]
   CMD ["/app/more_stuff/my_app"]
   

   If you care about such things, environment variables you set via this approach won’t be visible in docker inspect or a docker exec debug shell; but if you docker run -it ... sh they will be visible. This is a useful and important enough pattern that I almost always use CMD in my Dockerfiles unless I’m specifically trying to do first-time setup like this.

Answer 2

I found an alternative option that I like better:

Configure an ENTRYPOINT dockerfile step, that sources the file, and then runs the CMD received by argument:

ENTRYPOINT ["sh", "-c", "source /env.sh && \"$@\"", "-s"]

Answer 3

Although there is a good accepted answer and recommendation, there are other ways to pull this off including a method that is in a bit of a fashion more towards the original intent of the question to source from a bash script and set the value with ENV.

Additionally, someone might want to take this approach of sourcing a bash file and injecting the values into the environment if there is a use case that requires maintaining a common set of values across multiple images. The current answers don't provide a solution that covers this use case and allows for the injection of environment variables via ENV. Injecting values through the ENTRYPOINT precludes the ability to leverage these values in a subsequent RUN command within the same dockerfile.

Method 1 is geared more towards the original intent of the question to source the values from a bash script, whereas Method 2 provides a similar approach leveraging a common dockerfile.

Method 1 - Build Args and Scripts

Often times I tend to wrap my docker builds with build scripts to help standardize image builds (i.e. in an enterprise environment), even for simple use cases. Typically I add a --pull to docker builds that pull from a moving tag (e.g. lts, stable, etc.), then add custom build args when appropriate (e.g. varying the base or FROM of a docker image build).

When build scripts like this are already present, it might make more sense for some cases to leverage build args that are passed into the script, then set environment variables to these values if needed. Below is a quick example.

Dockerfile

FROM alpine:3.9.3

ARG test_val=
ENV TEST ${test_val}
CMD env

env.sh

export TEST=test123

build.sh

. env.sh
docker build --pull --build-arg test_val=${TEST} -t sandbox .

Now run the build script to build the docker image:

$ bash build.sh
Sending build context to Docker daemon  7.168kB
Step 1/4 : FROM alpine:3.9.3
3.9.3: Pulling from library/alpine
Digest: sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913
Status: Image is up to date for alpine:3.9.3
 ---> cdf98d1859c1
Step 2/4 : ARG test_val=
 ---> Running in 0e438f2b8a4b
Removing intermediate container 0e438f2b8a4b
 ---> a15edd0a5882
Step 3/4 : ENV TEST ${test_val}
 ---> Running in 16f83a6c6d8c
Removing intermediate container 16f83a6c6d8c
 ---> 28cdd3df03ec
Step 4/4 : CMD env
 ---> Running in 3057dd2682d6
Removing intermediate container 3057dd2682d6
 ---> e7afdb4eeff2
Successfully built e7afdb4eeff2
Successfully tagged sandbox:latest

Then run the docker image to see the environment variable set to the expected value:

$ docker run --rm sandbox
HOSTNAME=008e482ab3db
SHLVL=1
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST=test123
PWD=/

Method 2 - Base Dockerfile

Rather than maintaining these values in a bash script to source in the image, one could simply create a "common" dockerfile that sets all of these environment variables in a common base image. Then rather setting the FROM to the public image, instead set FROM to this common base image. Here's a quick example:

Dockerfile.base

FROM alpine:3.9.3

ENV TEST test123

Dockerfile1.frombase

FROM sandbox-base

# Some settings specific to this image.... example:
ENV MYIMAGE1 image1

CMD env

Dockerfile2.frombase

FROM sandbox-base

# Some different settings specific to this image....
ENV MYIMAGE2 image2

CMD env

Now build all the images:

docker build -f Dockerfile.base -t sandbox-base .
docker build -f Dockerfile1.frombase -t sandbox-image1 .
docker build -f Dockerfile2.frombase -t sandbox-image2 .

Then run the two target images for comparison:

$ docker run --rm sandbox-image1
HOSTNAME=6831172af912
SHLVL=1
HOME=/root
MYIMAGE1=image1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST=test123
PWD=/

$ docker run --rm sandbox-image2
HOSTNAME=fab3c588e85a
SHLVL=1
HOME=/root
MYIMAGE2=image2
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST=test123
PWD=/

Answer 4

I ended up do a multistep build of the dockerfile in a bash script:

1. Setup your Dockerfile to include everything up to the point where you need to source a file for environment variables.
2. In the docker file, source the environment variables and echo the environment to a file.

RUN  source $(pwd)/buildstepenv_rhel72_64.sh && source /opt/rh/devtoolset-8/enable && env | sort -u  > /tmp.env"  

3. Build the image with a tag: docker build -t ${image}_dev .
4. Run the image using the tag to get the environment variables and add them to the end of the docker file

docker run --rm  ${image}_dev cat /tmp.env | sed 's/$/"/;s/=/="/;s/^/ENV /'  >>  logs/docker/Dockerfile.${step}

5. Construct the remainder of your dockerfile.