Read Bash variables into a Python script

Question

I am running a bash script (test.sh) and it loads in environment variables (from env.sh). That works fine, but I am trying to see python can just load in the variables already in the bash script.

Yes I know it would probably be easier to just pass in the specific variables I need as arguments, but I was curious if it was possible to get the bash variables.

test.sh

#!/bin/bash
source env.sh

echo $test1

python pythontest.py

env.sh

#!/bin/bash

test1="hello"

pythontest.py

?
print test1 (that is what I want)

score 83 · Accepted Answer · answered Jul 02 '13 at 20:18

83

You need to export the variables in bash, or they will be local to bash:

export test1

Then, in python

import os
print os.environ["test1"]

answered Jul 02 '13 at 20:18

AMADANON Inc.

5,753
21
31

1

I am curious, is there an easy way to export all of the variables? – Tall Paul Jul 02 '13 at 20:26
1

@TallPaul The `os.environ` dictionary already contains all variables that can be seen by the program. – Bakuriu Jul 02 '13 at 20:44
16

`set -a; source env.sh; set +a`. The `-a` option instructs `bash` to export each new variable until you turn it off with `+a`. – chepner Jul 02 '13 at 20:47
5

Instead of using 'export', you can also set it in front of your command - in this case, it will apply ONLY to your command: 'test1=wibble python myscript' - NOTE: there is no semicolon in front of the 'python' - this is intentional. – AMADANON Inc. Aug 13 '17 at 06:30

Taejoon Byun · Answer 2 · 2017-11-30T21:23:13.990

There's another way using subprocess that does not depend on setting the environment. With a little more code, though.

For a shell script that looks like follows:

#!/bin/sh
myvar="here is my variable in the shell script"

function print_myvar() {
    echo $myvar
}

You can retrieve the value of the variable or even call a function in the shell script like in the following Python code:

import subprocess

def get_var(varname):
    CMD = 'echo $(source myscript.sh; echo $%s)' % varname
    p = subprocess.Popen(CMD, stdout=subprocess.PIPE, shell=True, executable='/bin/bash')
    return p.stdout.readlines()[0].strip()

def call_func(funcname):
    CMD = 'echo $(source myscript.sh; echo $(%s))' % funcname
    p = subprocess.Popen(CMD, stdout=subprocess.PIPE, shell=True, executable='/bin/bash')
    return p.stdout.readlines()[0].strip()

print get_var('myvar')
print call_func('print_myvar')

Note that both shell=True shall be set in order to process the shell command in CMD to be processed as it is, and set executable='bin/bash' to use process substitution, which is not supported by the default /bin/sh.

Kinda I have 20 envs and I need to source 20 file to get all of them. Could you help to write get_var from one source and not from each run? How is that possible to do? — Ivan Shelonik, Oct 09 '20 at 13:34
@IvanShelonik I guess you can change the bash code given in `CMD` variable to `echo` multiple variables line by line and run the same code. You'll also need to change the `return` statement as follows: `list(map(strip, p.stdout.readlines()))` so that the retrieved list of variables are returned. — Taejoon Byun, Oct 17 '20 at 17:54

score 4 · Answer 3 · answered Jul 02 '13 at 20:19

4

Assuming the environment variables that get set are permanent, which I think they are not. You can use os.environ.

os.environ["something"]

answered Jul 02 '13 at 20:19

John

13,197
7
51
101

1

add `set -a` before the `ource` command (on a separate line). This says, that from now on, any variable set should automatically be exported. – AMADANON Inc. Jul 02 '13 at 20:52
1

@TallPaul, it is as AMADANON Inc. says. However you can set variables for the current environment [from python](http://stackoverflow.com/a/1339904/322909). – John Jul 02 '13 at 21:06

score 1 · Answer 4 · answered Mar 17 '21 at 20:34

If you are trying to source a file, the thing you are missing is set -a. For example, to source env.sh, you would run:

set -a; source env.sh; set +a.

The reason you need this is that bash's variables are local to bash unless they are exported. The -a option instructs bash to export all new variables (until you turn it off with +a). By using set -a before source, all of the variables imported by source will also be exported, and thus available to python.

Credit: this command comes from a comment posted by @chepner as a comment on this answer.

score 0 · Answer 5 · answered Jan 25 '22 at 23:22

Two additional options, which may or may not help in your specific situation:

Option 1:

if env.sh only contains regular NAME=value, get python to read it (as a text file) instead.

This only applies if you control the format of env.sh, and env.sh doesn't contain any real shell commands, and you control the containing shell script.

Option 2:

In the shell script, once all the necessary variables are set, either save these to a file, or pipe them as stdin to your python script:

#!/bin/bash
source env.sh

echo $test1

set | python pythontest.py

or

#!/bin/bash
source env.sh

echo $test1

set > /tmp/$$_env
python pythontest.py --environment=/tmp/$$_env

You can then read the file (or stdin), and parse it into e.g. a dictionary.

Read Bash variables into a Python script

5 Answers5

Linked