1

I have a script called "script.sh," whose contents are:

#!/bin/sh
export A=5

I want to execute this script from within python (iPython actually) and read the variable 'A'.

import os
import subprocess

subprocess.call('./script.sh')
A=os.environ['A']

Unfortunately, this doesn't seem to work, giving me an error that A cannot be found. If I understand correctly, subprocess is actually running in a different shell than the one that os.environ queries. But then why can't I run something like:

subprocess.call('echo $A')

?

What should I change to make this work? In general, I just want to obtain the value of "A" from the script, preferably by executing the script through some form of shell (the actual script is quite long).

For some more info, the script will contain login credentials, so ideally I'd like a safe,minimalist way of accessing their values.

Alex R.
  • 1,397
  • 3
  • 18
  • 33
  • 1
    There is nothing you can do to make this work. A subshell can't set environment variables in its parent. – Daniel Roseman Aug 11 '17 at 19:19
  • @DanielRoseman: Right, I've just added an edit. Is there a way I could access it from the subprocess shell? That's the part I don't get – Alex R. Aug 11 '17 at 19:20
  • Yes, you're successfully setting the environment variable in the shell spawned by `subprocess.call`, which then disappears right after your script runs. – kindall Aug 11 '17 at 19:20
  • The subprocess is using a different instance of the shell each time. – kindall Aug 11 '17 at 19:21
  • `os.environ` is captured the first time the `os` is imported. Even if it wasn't the subprocess, it wouldn't work. https://docs.python.org/3/library/os.html#os.environ – Eugene Sh. Aug 11 '17 at 19:21
  • @kindall: Is there a way I can immediately return the values of the subprocess call? I only care about obtaining the value of A, and have no need of actually setting A in the enviornment shell. – Alex R. Aug 11 '17 at 19:22
  • Maybe you simply want to access stdout instead? See https://stackoverflow.com/questions/1996518/retrieving-the-output-of-subprocess-call – Tomalak Aug 11 '17 at 19:22
  • Your "A" not an variable ! `$` ? – dsgdfg Aug 11 '17 at 20:26
  • I also believe [Execute bash script that defines environment variables using Python](https://stackoverflow.com/questions/43500328/execute-bash-script-that-defines-environment-variables-using-python) to be pertinent. – Charles Duffy Aug 11 '17 at 20:55
  • @CharlesDuffy: I'm not entirely sure I understand how the marked-duplicate question answers this one. Would I run: subprocess.Popen(['. "$1"; shift; exec "$@"', "_", "./script.sh", "echo", "$A"], shell=True) (that doesn't seem to work) – Alex R. Aug 11 '17 at 21:11
  • @AlexR., you're looking at the wrong section -- adopting parts that are for when `script.sh` is something that *uses* the variable itself, as opposed to transporting its value back to the parent. The answer *does* distinguish between its various sections, and which is useful in which circumstance. – Charles Duffy Aug 11 '17 at 21:28
  • @AlexR., ...what you want here is emitting a NUL-delimited stream from the subprocess and parsing it in the parent, which is what... well, really, everything but the top 1/4 of the linked question's accepted answer discusses. – Charles Duffy Aug 11 '17 at 21:31
  • Take a look at `subprocess.Popen(['set -a; . "$1"; cat /proc/self/environ', "_", "./script.sh", shell=True, stdout=subprocess.PIPE).communicate()[0].split('\0')`; you'll see that `A=5` is among the members of that list. The linked answer also covers practices that'll work where `/proc/self/environ` (being a Linuxism) isn't available. – Charles Duffy Aug 11 '17 at 21:38

2 Answers2

0

You need to source the script in the subshell (so it sets the variable in the same shell process), then echo the variable in that subshell:

a = subprocess.check_output('source ./script.sh; echo "$A"', shell=True)

Then you can read from the pipe to get the value of the variable.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • Thanks for your answer. I'm not an expert at this but from what I understand, shell=true and stdout=PIPE is discouraged in the answers I've seen. Specifically, if my script contains credentials, is this still a good solution? – Alex R. Aug 11 '17 at 19:28
  • I changed it from `subprocess.call` to `subprocess.check_output`. Is that still discourged? If so, you should use `Popen()` and `communicate`. – Barmar Aug 11 '17 at 19:29
  • I don't see how credentials in the script are relevant to this. – Barmar Aug 11 '17 at 19:29
  • If it contains credentials, you shouldn't be `export`ing them at all unless you're sure your OS protects environment variables from being read out of process metadata (modern systems do, a great many old ones don't). – Charles Duffy Aug 11 '17 at 20:34
  • @CharlesDuffy I thought he meant there were credentials elsewhere in the script, not that the value of `A` is credentials, but you're probably right. – Barmar Aug 11 '17 at 20:36
0

The trick will be to spawn a new shell and tell it to both interpret your script's code, and can print out its environment in a way that Python can read it. A one-liner:

In [10]: subprocess.check_output(["bash", "-c", "source ./script.sh; env"])
Out[10]: '...\nA=5\n...'

What's happening: In general, environment variables are set at the beginning of a program, and any subprocesses can't modify their parent's environment; it's a sort of sandbox. But source is a bash builtin where bash says "instead of spawning script.sh as a new (sub-)subprocess which couldn't modify my environ, run the lines of code as myself (bash) and modify my environ accordingly for future commands". And env is tacked on so that bash prints the environment separated by newlines. check_output simply grabs that output and brings it back into Python.

(As a side note, that source command is what you use to update a shell to use a certain virtualenv: source my_project/bin/activate. Then the $PATH and other variables of your current shell are updated to use the virtualenv python and libraries for the rest of that session. You can't just say my_project/bin/activate since it would set them in a subshell, doing nothing :))

btown
  • 2,273
  • 3
  • 27
  • 38
  • Environment variables can contain literal newlines. This thus isn't a safe way of representing and parsing them. – Charles Duffy Aug 11 '17 at 20:33
  • (Consider if an attacker were able to set an environment variable, say via CGI's automatic header->variable mechanism, with a value akin to that created by `CGI_X_evil=$'hello\nLD_PRELOAD=/tmp/exploit.so'` -- if code parsing the output of `env` saw `LD_PRELOAD=/tmp/exploit.so` and then updated your Python program's environment with that, future programs that your Python tool invoked would be running arbitrary code). – Charles Duffy Aug 11 '17 at 20:57
  • @CharlesDuffy Generally, a good point. OP was only trying to update their Python program's environment as a means to an end; as they explained, all they want to do is read the environment created by the script, and get it as a string in Python. Nobody's talking about using that output as the environment for any future program, or the current program; just extracting trusted environment variables. – btown Aug 14 '17 at 02:03