45

I'm working on Debian Jessie with Python 2. Why can't Python's environ see environment variables that are visible in bash?

# echo $SECRET_KEY
xxx-xxx-xxxx
# python
>>> from os import environ
>>> environ["SECRET_KEY"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.virtualenvs/prescribing/lib/python2.7/UserDict.py", line 23, in __getitem__
    raise KeyError(key)
KeyError: 'SECRET_KEY'

I set these environment variables using /etc/environment - not sure if that's relevant:

SECRET_KEY=xxx-xxx-xxx

I had to run source /etc/environment to get bash to see them, which I thought was strange.

UPDATE: printenv SECRET_KEY produces nothing, so I guess SECRET_KEY is a shell not an environment variable.

Richard
  • 62,943
  • 126
  • 334
  • 542
  • 2
    That's not an environment variable; it is just a shell variable until you `export` it. – Jonathan Leffler Aug 13 '15 at 16:40
  • Really? I thought `/env/environment` was for setting environment variables - is this incorrect? – Richard Aug 13 '15 at 16:44
  • 1
    Using `source /env/environment` only sets variables in the shell, not environment variables accessible by child processes. – Martijn Pieters Aug 13 '15 at 16:46
  • Yes, no, maybe — yes, it is incorrect (because they're only environment variables if you export them); no, it is correct that `/etc/environment` might be used to set environment variables (though it isn't a standard file, AFAIK, but it is referenced in Ubuntu, it seems); maybe, because it depends what you write in the file (or what is written in the file if you don't have control over its contents). – Jonathan Leffler Aug 13 '15 at 16:46
  • What OS is this on? I've never seen `/env/environment` used, not for shells at any rate. – Martijn Pieters Aug 13 '15 at 16:47
  • Typo sorry, should have been `/etc/environment`. Fixed in question. – Richard Aug 13 '15 at 16:50
  • I was trying to follow https://help.ubuntu.com/community/EnvironmentVariables#A.2Fetc.2Fenvironment - I'm now confused about what I should be putting in that file. Should it be `foo=bar` or `export foo=bar`? I had `foo=bar` but perhaps that's incorrect. – Richard Aug 13 '15 at 16:51
  • Since the referenced file has a restriction ([Note: Variable expansion does not work in /etc/environment](https://help.ubuntu.com/community/EnvironmentVariables#A.2Fetc.2Fenvironment)), it is not clear how that is processed. It might be handled differently from normal files, somehow. However, if you use `source /etc/environment` and it does not contain `export` directives, then they aren't environment variables and won't be exported by default. Look up the [`set -k` and `set -a`](http://www.gnu.org/software/bash/manual/bash.html#The-Set-Builtin) options. – Jonathan Leffler Aug 13 '15 at 16:53

1 Answers1

64

You need to export environment variables for child processes to see them:

export SECRET_KEY

Demo:

$ SECRET_KEY='foobar'
$ bin/python -c "import os; print os.environ.get('SECRET_KEY', 'Nonesuch')"
Nonesuch
$ export SECRET_KEY
$ bin/python -c "import os; print os.environ.get('SECRET_KEY', 'Nonesuch')"
foobar

You can combine the setting and exporting in one step:

export SECRET_KEY=xxx-xxx-xxxx

Note that new variables in /etc/environment do not show up in your existing shells automatically, not until you have a new login. For a GUI desktop, you'll have to log out and log in again, for SSH sessions you'll have to create a new SSH login. Only then will you get a new tree of processes with the changes present. Using source /etc/environment only sets 'local' variables (the file is not a script). See How to reload /etc/environment without rebooting? over on Super User.

Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • I was trying to follow the instructions in https://help.ubuntu.com/community/EnvironmentVariables#A.2Fetc.2Fenvironment - should I use `export foo-bar` instead of `foo=bar` in /etc/environment? – Richard Aug 13 '15 at 16:46
  • 2
    It's worth noting that `os.environ` is only loaded the first time the `os` module is imported. So if you call a `subprocess` to execute a bash script, when it returns to the python script it won't be able to see the new environment variables that were possibly exported in your bash script. – amza Aug 13 '15 at 16:48
  • 1
    @Richard: in your question you named it `/env/environment`, not `/etc/environment`.. I'm not *that* familiar with Ubuntu, I'll have a look as to how the OS is using those. Did you restart at all after making changes to the file? – Martijn Pieters Aug 13 '15 at 16:48
  • @amza: a subprocess won't be able to set variables for the parent process *anyway*. This has nothing to do with `os.environ` being loaded when first imported (the mapping is actually built when *Python starts*). The process environment can only be altered by the current process or a *parent* process, and `subprocess` never creates such a process, so the distinction is entirely moot. – Martijn Pieters Aug 13 '15 at 16:49
  • 1
    @Richard: a quick experiment shows that changes in `/etc/environment` only apply to *new logins*; after editing I didn't see the change in new subshells, but use `ssh localhost` and those new login sessions do see the new variable. So I suspect that this is used by PAM to set up the new user environment. – Martijn Pieters Aug 13 '15 at 16:55
  • Didn't work for me. I exported it but still get the error. – Ariel Sep 03 '18 at 11:27
  • @Ariel: that's not something that we can help you with, though. I can't see how you exported your shell variables, and what software is not seeing that variable set or how you run your software. Exported shell variables are visible to *child processes only*, not to processes started from a different shell or parent process, for example. – Martijn Pieters Sep 03 '18 at 11:30
  • @MartijnPieters it was an array variable and it turns out those cannot be exported at all. I was trying to read it from a Python shell, like the OP. – Ariel Sep 03 '18 at 11:37