How can I access Docker set Environment Variables From a Cron Job

Question

I've recently tried running a cron job from within a linked docker container and run into an issue. My main docker container is linked to a postgres container and its port number is set as an environment variable by docker upon the containers creation. This environment variable is not set in ~/.profile or any other source file that I could load when running my cron job. How can I then access these environment variables from my cron job?

Thanks!

This post seems to answer the question http://stackoverflow.com/questions/26822067/running-cron-python-jobs-within-docker — user2915097, Jan 05 '15 at 12:04

Tim Schruben · Answer 1 · 2017-01-30T14:27:10.970

122

I ran into this same problem. I have a docker container that runs cron to execute some shell scripts periodically. I too had a hard time finding out why my scripts would run fine when I manually executed them inside the container. I tried all the tricks of creating a shell script that would run first to set the environment, but they never worked for me (most likely I did something wrong). But I continued looking and found this and it does work.

Setup a start or entry point shell script for your cron container
Make this the first line to execute printenv | grep -v "no_proxy" >> /etc/environment

The trick here is the /etc/environment file. When the container is built that file is empty, I think on purpose. I found a reference to this file in the man pages for cron(8). After looking at all the versions of cron they all elude to an /etc/? file that you can use to feed environment variables to child processes.

Also, note that I created my docker container to run cron in the foreground, cron -f. This helped me avoid other tricks with tail running to keep the container up.

Here is my entrypoint.sh file for reference and my container is a debian:jessie base image.

printenv | grep -v "no_proxy" >> /etc/environment

cron -f

Also, this trick worked even with environment variables that are set during, docker run commands.

edited Jan 30 '17 at 14:27

answered Jan 30 '17 at 14:19

Tim Schruben

1,535
1
12
10

26

printenv > /etc/environment was the only thing that worked for me after 4 days och experimenting with various options ! Thanks! – Deko Apr 13 '17 at 19:07
1

Using "/etc/environment" worked for me, however please note that reading this file is not standard behavior for cron - it only works if you have the pam cron module installed, please see https://askubuntu.com/a/700126 – llmora Nov 14 '17 at 15:58
4

Watch out for the limitation of using `/etc/environment` file as there is no possibility to escape the comment sign `#`, so you can not use it in the values of the variables there... – AuHau Jan 18 '18 at 19:35
6

It might be worth noting that the `... | grep -v " no_proxy` part is specific to certain environments and it's just a red herring for most. As @Deko says, for most situations the magic is simply `printenv > /etc/environment`. (Also, using ">>" would append to any existing /etc/environment file but I've never seen one that's not empty to start. Does anyone have a different experience??) – Tom Wilson May 06 '20 at 17:00
2

Please be aware that `>>` will cause add cron line as many times as docker container restart, better just `>` – Javier Feb 19 '21 at 07:25
Hi @Javier, that is true if the file is mounted in. Since my file was part of the image the changes are only saved in the scratch layer that is created when all the layers are merged together in the container start. Using the `>` will just overwrite any changes you already have, so using `>` and `>>` all depends on your situation. – Tim Schruben Feb 22 '21 at 14:10
@TimSchruben very interesting, thanks for sharing your wisdom :) – Javier Feb 23 '21 at 15:24
Perfect! The simplest and workable solution. – Timofey Bugaevsky Apr 23 '21 at 23:26
Thank you so much. Even setting the env variables manually at the top of my crontab entries didn't solve this problem for me but this is so much more elegant and it actually works! – JackofSpades Nov 10 '21 at 04:40
@TomWilson what is the environment where "no_proxy" is a problem? NO_PROXY still gets through so I assume it's not just when you don't want a proxy. – Chris Feb 16 '22 at 20:41
@Chrus, you got me. I've never seen no_proxy in the wild. I'm assuming it's an environment setting that pops up in some configurations. `printenv > /etc/environment` does the trick for us. – Tom Wilson Feb 17 '22 at 04:43
How can one run the `entrypoint.sh` file from the dockerfile? – Oliver Angelil Jun 28 '22 at 08:14
@OliverAngelil, not sure exactly what you mean. But when you build a docker image there is a stanza `ENTRYPOINT` where you specify the path to your `entrypoint.sh` file. When you docker image runs the daemon will execute that script for you. If you want to test it by getting a shell session inside your container, then run your image setting the `--entrypoint=""` and that will set it back to default. You can the run you string manually. – Tim Schruben Jul 01 '22 at 15:25
Doesn't seem to work on debian bullseye – Quentin Sommer Oct 27 '22 at 21:21

Alex Fedulov · Answer 2 · 2021-06-18T15:01:32.727

50

I would recommend using declare to export your environment and avoid escaping issues. Can be used in CMD or ENTRYPOINT or directly in a wrapper script which might be called by one of them:

Grep -v takes care of filtering out read-only variables.

You can later easily load this environment like this:

SHELL=/bin/bash
BASH_ENV=/container.env
* * * * * root /test-cron.sh

edited Jun 18 '21 at 15:01

answered Feb 06 '18 at 20:03

Alex Fedulov

1,442
17
26

1

Thanks a lot, this works much better than the first answer with printenv that doesn't escape anything – Emmanuel Courreges Aug 31 '18 at 09:28
This should be the accepted solution. Using the "/etc/environment" file as others have mentioned in this thread isn't as good idea as it first seems unfortunately (ref https://unix.stackexchange.com/questions/97736/escape-hash-mark-in-etc-environment) :/ – Even André Fiskvik Jun 17 '19 at 20:17
2

I think this is the right answer too, but I haven't used the "-v" of the "grep", I've preferred put the specific variables that I will use, like this: `declare -p | grep -E 'VAR1|VAR2|VAR3|ETC' > /container.env` – Thiago G. Alves Jun 24 '19 at 18:22
2

Alex Fedulov's regex is too greedy. It will filter out more than the intended variable declarations (it would f.ex. filter out a variable named `FLUID`). I suggest using the expression `declare -p | grep -Ev '^declare -[[:alpha:]]*r'` which should filter out all the readonly declarations. – Tomáš Pospíšek Oct 15 '19 at 10:34
2

@TomášPospíšek or maybe amend the pattern to respect word boundaries?: `grep -Ev '\b(BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID)\b'`, or even `'\b(BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID)='` – laur Sep 30 '20 at 13:22
1

Most elegant and bulletproof solution I found Of course, apart of fixing this in the container runtime. BTW - anybody knows how the envs are passed between container runtime and linux OS? The env is not part of e.g. `/etc/profile` and I'm scratching my head how it is working. – Jarek Dec 12 '22 at 10:54

score 6 · Answer 3 · answered Jan 08 '16 at 14:13

One can append the system environment variables to the top of a crontab file by using wrapper shell script to run the cron daemon. The following example is from CentOs 7,

In the Dockerfile

COPY my_cron /tmp/my_cron
COPY bin/run-crond.sh run-crond.sh
RUN chmod -v +x /run-crond.sh
CMD ["/run-crond.sh"]

run_cron.sh:

#!/bin/bash

# prepend application environment variables to crontab
env | egrep '^MY_VAR' | cat - /tmp/my_cron > /etc/cron.d/my_cron

# Run cron deamon
# -m off : sending mail is off 
# tail makes the output to cron.log viewable with the $(docker logs container_id) command
/usr/sbin/crond -m off  && tail -f /var/log/cron.log

This is based on a great blog post somewhere, but I lost the link.

score 6 · Answer 4 · answered Jan 15 '20 at 10:10

6

inspired by @Kannan Kumarasamy answer:

  for variable_value in $(cat /proc/1/environ | sed 's/\x00/\n/g'); do
    export $variable_value
  done

I can't state for sure, what process with pid1 is and that its stable during lifetime of OS. but as it is the first process to be run, inside a container i guess we can take it for granted it is a process with desired env vars set. take all of this with pich of salt unless some linux/docker docs proves this is completely ok.

answered Jan 15 '20 at 10:10

Ondřej Želazko

648
7
16

Read permission only for root. And one minor note, shellcheck warns "Useless cat" and suggests using `$(sed 's/\x00/\n\g' < /proc/1/environ)` instead. – chrisinmtown Jan 27 '22 at 18:38
actually, you trim down further: `sed 's/\x00/\n/g' /proc/1/environ` sed takes a file as input and spits out the changes (unless specified otherwise, with -i inplace for example) – user3054986 May 24 '23 at 06:25

Kannan Kumarasamy · Answer 5 · 2019-04-02T02:46:31.563

4

You can run the following command

. <(xargs -0 bash -c 'printf "export %q\n" "$@"' -- < /proc/1/environ)

This is exceptionally works when you have the environment variables with special characters like ' " =

edited Apr 02 '19 at 02:46

answered Apr 02 '19 at 02:27

Kannan Kumarasamy

41
3

well. have a +1 from me. but could you point us to some docs that proves pid 1 is always the process that has the original environment variables? – Ondřej Želazko Jan 15 '20 at 09:30
this one alos work with any special character in your envrionment variable. `eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)` – Amul Dec 02 '20 at 15:35

score 3 · Answer 6 · answered Jan 29 '16 at 16:01

3

The environment is set, but not available to the cron job. To fix that, you can do these two simple things

1) Save the env to a file in your ENTRYPOINT or CMD

CMD env > /tmp/.MyApp.env && /bin/MyApp

2) Then read that env into your cron command like this:

0 5 * * * . /tmp/.MyApp.env; /bin/MyApp

answered Jan 29 '16 at 16:01

Mark Riggins

471
4
5

3

This won't work when env values contain characters such as whitespaces or pipes – Laurens Rietveld Feb 16 '16 at 20:47
You'll need to properly escape env vars with whitespaces or pipes. – GDorn Mar 13 '16 at 21:22
does `.env` file has have executable ? – Chang Zhao Oct 03 '21 at 10:05

score 3 · Answer 7 · answered Dec 02 '20 at 15:29

You should export your environment variable before you run cronjobs.

Other solutions are fine but they will fail when there are any special characters in your environment variable.

I have found the solution:

eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)

score 0 · Answer 8 · edited May 23 '17 at 12:25

0

To escape any weird characters that could break your script, and according to reasoning from Mark's answer, add this line to your entrypoint.sh:

env | sed -r "s/'/\\\'/gm" | sed -r "s/^([^=]+=)(.*)\$/\1'\2'/gm" \ > /etc/environment

This way, if you have any variable like affinity:container==My container's friend, it will be converted to affinity:container='=My container\'s friend, and so on.

edited May 23 '17 at 12:25

Community

1
1

answered Mar 30 '17 at 10:00

Yajo

5,808
2
30
34

1

Even this will fail if there's a # charater in one of the environment variables. See https://unix.stackexchange.com/questions/97736/escape-hash-mark-in-etc-environment – Even André Fiskvik Jun 17 '19 at 20:14

How can I access Docker set Environment Variables From a Cron Job

8 Answers8

Linked

Related