Python script can't access some environment variables

Question

I am building Docker Image FROM node:8.9.3-alpine (which is Debian) and then running it as usual and passing parameters like this:

docker run -dt \
-e lsRegion=${bamboo_lsRegion} \
-e lsCluster=${bamboo_lsCluster} \

Then inside that container I am exporting some variables and when I echo them, I can see proper value

export lsEnv=${lsEnv:-'dev'}

Later in scripts I run python script and when I run the print(os.environ) I can see all the variables from docker run like lsRegion but I do not see the newly exported one like lsEnv.

I already found and tried to solve with this: Python: can't access newly defined environment variables by calling the source ~/.bashrc but I cannot find that file.

I have tried

~/.bashrc
/etc/bash.bashrc
/root/.bashrc

But neither of those exist (also does not know if this solve my problem) and it ends with this error message /app/deploy.sh: source: line 16: can't open '/root/.bashrc'

More reproducible example:

Dockerfile

FROM node:8.9.3-alpine

RUN apk add --no-cache \
    python \
    py-pip \
    ca-certificates \
    openssl \
    groff \
    less \
    bash \
    curl \
    jq \
    git \
    zip \
    build-base \
  && pip install --no-cache-dir --upgrade pip awscli \
  && aws configure set preview.cloudfront true

ENV TERRAFORM_VERSION 0.11.10

RUN wget -O terraform.zip https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && \
    unzip terraform.zip -d /usr/local/bin && \
    rm -f terraform.zip

RUN apk -v --update add python py-pip
RUN pip install --upgrade awscli
RUN pip install --upgrade boto3

COPY ./build.variables /app/build.variables
COPY ./aws/taskdef/template.json /app/template.json
COPY ./deploy.sh /app/deploy.sh
COPY ./deploy.py /app/deploy.py
COPY ./terraform /app/terraform

CMD ["sh", "/app/deploy.sh"]

deploy.sh

#!/bin/bash -x

cd /app/terraform
./run-terraform.sh
cd ..
python /app/deploy.py

terraform/run-terraform.sh

   ...
   export lsEnv="NotThere"
   ...

python script

#!/usr/bin/env python
import os
print(os.environ)

The print will show lsRegion or lsCluster but it will not show the lsEnv

Create a `~/.bashrc ` file yourself and add your exports to it. Then source this file. — max, Nov 13 '18 at 14:35
@max And if I dont have it, what happens with exported variables? They are just living in a memory? — libik, Nov 13 '18 at 14:36
If `.bashrc` contains calls to `export`, then sourcing it does the same thing as running the same `export` commands manually. — chepner, Nov 13 '18 at 14:39
How *exactly* do you start the container, run the `export` command, and run the Python script. It sounds like you aren't starting the Python script from the environment you think you are. — chepner, Nov 13 '18 at 14:41
You probably call the Python script from another context than where you exported the variables. So adding them to `~/.bashrc` should fix that. — max, Nov 13 '18 at 14:41
Can you share your whole Dockerfile? Why/how are you running a Python script in a Node-based image? — David Maze, Nov 13 '18 at 15:19
The problem has nothing to do with Docker. You are *executing* `run-terraform.sh`, not sourcing it, so your definition of `lsEnv` disappears as soon as `run-terraform.sh` exits. — chepner, Nov 14 '18 at 12:59

score 1 · Accepted Answer · answered Nov 14 '18 at 13:30

1

Inside deploy.sh, you need to source run-terraform.sh if you want to affect the environment of the process that runs deploy.py, rather than the environment created for the process that runs run-terraform.sh.

#!/bin/bash -x

cd /app/terraform
source ./run-terraform.sh
cd ..
python /app/deploy.py

(You could also use . ./run-terraform.sh; source is a more readable bash synonym for the POSIX . command, but . is necessary if you are using some other POSIX-compliant shell that doesn't support source.)

answered Nov 14 '18 at 13:30

chepner

497,756
71
530
681

I have encounter another problem. When I do this, it fails with `./run-terraform.sh: line 41: syntax error: bad substitution` which is `terraform apply -var "environment=$lsSpecificEnv" -var "environment_vpc=${lsEnv^}" -var "region=$lsRegion" -var "service=$SERVICE_NAME" -var "path_pattern=/v*/$SERVICE_NAME" -auto-approve`. I dont understand whats going on as both files (deploy and run-terraform) starts with `#!/bin/bash -x` anyway. Does `source` change the execution? – libik Nov 14 '18 at 13:48
I can confirm that this `${lsEnv^}` does work without `source` when running the script and when I run it with `source` it stops working with that error – libik Nov 14 '18 at 14:17
You are either using an older version (pre-4.0) version of `bash`, or more likely `sh` does not refer to `bash` in Alpine Linux, but some other POSIX shell that doesn't support `${...^}` syntax. (In that case, `source` itself probably won't work, but the shell doesn't get as far as trying to run the `source` command before the parameter expansion fails.) – chepner Nov 14 '18 at 14:17
With `CMD ["sh", "/app/deploy.sh"]`, the shebang is irrelevant; `sh` is used to run the script, ignoring the shebang. – chepner Nov 14 '18 at 14:19
actually I used source and it works (look to my answer), when I dont use source the line is executed properly, but if its behind source, it does not. Really weird. However I solve it with `CMD ["bash", "/app/deploy.sh"]` thanks for noticing that. – libik Nov 14 '18 at 16:05

score 0 · Answer 2 · answered Nov 14 '18 at 11:15

I solve it by calling this command in terraform/run-terraform.sh for each environment variable I will need in python script:

echo "export lsTargetGroup=$lsTargetGroup" >> ~/.bashrc

And then in deploy.sh I just add source ~/.bashrc before calling python script

Python script can't access some environment variables

2 Answers2