7

I want to set an environmental variable in linux terminal through a python script. I seem to be able to set environmental variables when using os.environ['BLASTDB'] = '/path/to/directory' .

However I was initially trying to set this variable with subprocess.Popen with no success.

import subprocess
import shlex

cmd1 = 'export BLASTDB=/path/to/directory'
args = shlex.split(cmd1)
p = subprocess.Popen(args, stdout=subprocess.PIPE).communicate()

Why does subprocess.Popen fail to set the environmental variable BLASTDB to '/path/to/directory'?

NOTE: This also fails when using:

import os
os.system('export BLASTDB=/path/to/directory')
HelloIAreTheVictors
  • 105
  • 1
  • 2
  • 6
  • Note: `Popen` does not execute a shell. `export` is a shell built-in, so you'd get a "file not found error" if you try to run that command. Use `shell=True` to execute *shell* commands (and be aware of its security risks!!!) – Bakuriu Jun 19 '14 at 17:43
  • From my testing it seems that even when adding `shell=True` to the command it does not actually change the environment variable from which my python script was executed, nor does it change the environment variable for the python subshell that was created upon execution. This may only change the environment for this one specific subprocess, which (without the `env` addition) will not allow me to execute a command to the terminal with a modified environment for that subprocess. – HelloIAreTheVictors Jun 20 '14 at 16:00
  • I never stated that `shell=True` would solve your problem (otherwise I'd posted that as an answer). Using `shell=True` would be a *prerequisite* to be able to run shell commands. Without it you'll receive errors because `Popen` isn't able to find the executable. – Bakuriu Jun 20 '14 at 16:47
  • Agreed, I was trying to clarify for future visitors in regards to my specific issue. – HelloIAreTheVictors Jun 20 '14 at 18:56

2 Answers2

9

Use the env parameter to set environment variables for a subprocess:

proc = subprocess.Popen(args, stdout=subprocess.PIPE,
                        env={'BLASTDB': '/path/to/directory'})

Per the docs:

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of inheriting the current process’ environment, which is the default behavior.

Note: If specified, env must provide any variables required for the program to execute. On Windows, in order to run a side-by-side assembly the specified env must include a valid SystemRoot.

os.environ can used for accessing current environment variables of the python process. If your system also supports putenv, then os.environ can also be used for setting environment variables (and thus could be used instead of Popen's env parameter shown above). However, for some OSes such as FreeBSD and MacOS, setting os.environ may cause memory leaks, so setting os.environ is not a robust solution.

os.system('export BLASTDB=/path/to/directory') runs a subprocess which sets the BLASTDB environment variable only for that subprocess. Since that subprocess ends, it has no effect on subsequent subprocess.Popen calls.

unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • I believe the OP is trying to set the environment variable for the executing terminal /through/ Python. Your answer will set the environment variable only for the subprocess itself. – Dologan Jun 19 '14 at 17:36
  • @Dologan: Even if that's true, the only purpose for setting the environment variable would be to affect subsequent subprocess calls, in which case using `env` is the answer. – unutbu Jun 19 '14 at 17:42
  • @unutbu: From my testing `env` works great and temporarily sets the environmental variable `BLASTDB` for that specific subprocess, and everything moves along swimmingly. Thanks! For my purposes this is sufficient. os.environ is nice because it seems to set the environmental variable for the whole python subshell, and specifying `env` in each `subprocess.Popen` is not required. – HelloIAreTheVictors Jun 20 '14 at 15:49
  • Actually, according to the documentation of [`os.environ`](https://docs.python.org/2/library/os.html#os.environ), it *can* be used to modify the environment of the current process (if the OS provides a `putenv` syscall). – Bakuriu Jun 20 '14 at 16:51
  • @Bakuriu, on some OSes [setting environ causes memory leaks](https://docs.python.org/2/library/os.html#os.putenv), so it is not a reliable solution. But thank you for the comment; I've updated my answer to clarify this. – unutbu Jun 20 '14 at 18:01
  • Can we also add the currrent env variables along the one we are adding such as `PATH`? Because when I pass a binary as a comment, I had to provide its full path (`which `), if I pass `env={}` parameter @unutbu – alper Dec 24 '20 at 17:02
1

As far as I know, you cannot really modify the executing process' environment from a subprocess or subshell, be it Python or bash itself. Environment variables are specific to the particular process you are on (at least on Unix, which you seem to be using).

Any child process spawned will usually inherit that environment, but only a copy of it. For instance, if you run bash from inside your terminal session and export a new environment variable, once you exit that subshell, your original shell will be untouched. Running Python is no different.

Dologan
  • 4,554
  • 2
  • 31
  • 33
  • So far I have not been able to modify the executing process' environment. For me, I only need temporary adjustment for specific subprocess commands. However, I was curious if this was possible. Thanks for the info. – HelloIAreTheVictors Jun 20 '14 at 15:54
  • It is possible via the env keyword arg to subprocess.Popen as shown in the answer from unutbu – Paul Dec 11 '14 at 14:06
  • The original question suggests that the OP wants to set the environment variable for the linux terminal itself, not just for a subprocess executed from within Python itself. In this case, my answer is still, to the best of my knowledge, valid and correct. – Dologan Dec 12 '14 at 14:58