3

I have seen plenty examples of running a python script from inside a bash script and either passing in variables as arguments or using export to give the child shell access, I am trying to do the opposite here though.

I am running a python script and have a separate file, lets call it myGlobalVariables.bash

myGlobalVariables.bash:

foo_1="var1"    
foo_2="var2"   
foo_3="var3"  

My python script needs to use these variables.

For a very simple example:

myPythonScript.py:

print "foo_1: {}".format(foo_1)

Is there a way I can import them directly? Also, I do not want to alter the bash script if possible since it is a common file referenced many times elsewhere.

Bagelstein
  • 202
  • 4
  • 16
  • you mean you want to parse the bash file from python? won't be easy. Does your script contain commands besides the variables declaration? – Jean-François Fabre Nov 29 '16 at 16:25
  • It does, but not many. Was trying to avoid parsing, but I could definitely do it if that's the only option. – Bagelstein Nov 29 '16 at 16:27
  • Are the variables that you want to reference all simple assignments? i.e `var_name=var_val` - such that there's no shell expansion, command substitution, etc. – theorifice Nov 29 '16 at 16:29

4 Answers4

3

If your .bash file is formatted as you indicated - you might be able to just import it direct as a Python module via the imp module.

import imp
bash_module = imp.load_source("bash_module, "/path/to/myGlobalVariables.bash")
print bash_module.foo_1
theorifice
  • 670
  • 3
  • 9
  • " imp.load_source(name, pathname[, file]) Load and initialize a module implemented as a Python source file and return its module object.". No mention of bash... – Jean-François Fabre Nov 29 '16 at 16:30
  • According to the OP's description of `myGlobalVariables.bash` - it could be interpreted as a Python file. – theorifice Nov 29 '16 at 16:31
  • 1
    nice hack. Could just work... an `export` and you're history :) – Jean-François Fabre Nov 29 '16 at 16:31
  • Hmm so I tried this and got: SyntaxError: EOL while scanning string literal Which I think is related to formatting errors. The formatting is not quite as simple as I had indicated in my example, there are a few lines that don't follow this. – Bagelstein Nov 29 '16 at 23:17
  • In that case, you're just going to have to parse the contents. Assuming that all variable assignments simple you could probably just use a regex. – theorifice Nov 30 '16 at 14:21
  • Looks like it is python2 example. I got DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses – Ashark Apr 26 '21 at 23:41
1

You can also use os.environ:

Bash:

#!/bin/bash
# works without export as well
export testtest=one

Python:

#!/usr/bin/python
import os
os.environ['testtest']  # 'one'
Marat
  • 15,215
  • 2
  • 39
  • 48
0

I am very new to python, so I would welcome suggestions for more idiomatic ways to do this, but the following code uses bash itself to tell us which values get set by first calling bash with an empty environment (env -i bash) to tell us what variables are set as a baseline, then I call it again and tell bash to source your "variables" file, and then tell us what variables are now set. After removing some false-positives and an apparently-blank line, I loop through the "additional" output, looking for variables that were not in the baseline. Newly-seen variables get split (carefully) and put into the bash dictionary. I've left here (but commented-out) my previous idea for using exec to set the variables natively in python, but I ran into quoting/escaping issues, so I switched gears to using a dict.

If the exact call (path, etc) to your "variables" file is different than mine, then you'll need to change all of the instances of that value -- in the subprocess.check_output() call, in the list.remove() calls.

Here's the sample variable file I was using, just to demonstrate some of the things that could happen:

foo_1="var1"
foo_2="var2"
foo_3="var3"
if [[ -z $foo_3 ]]; then
    foo_4="test"
else
    foo_4="testing"
fi
foo_5="O'Neil"
foo_6='I love" quotes'
foo_7="embedded
newline"

... and here's the python script:

#!/usr/bin/env python

import subprocess

output = subprocess.check_output(['env', '-i', 'bash', '-c', 'set'])
baseline = output.split("\n")

output = subprocess.check_output(['env', '-i', 'bash', '-c', '. myGlobalVariables.bash; set'])
additional = output.split("\n")

# these get set when ". myGlobal..." runs and so are false positives
additional.remove("BASH_EXECUTION_STRING='. myGlobalVariables.bash; set'")
additional.remove('PIPESTATUS=([0]="0")')
additional.remove('_=myGlobalVariables.bash')
# I get an empty item at the end (blank line from subprocess?)
additional.remove('')

bash = {}
for assign in additional:
        if not assign in baseline:
                name, value = assign.split("=", 1)
                bash[name]=value
                #exec(name + '="' + value + '"')

print "New values:"
for key in bash:
  print "Key: ", key, " = ", bash[key]

Another way to do it:

Inspired by Marat's answer, I came up with this two-stage hack. Start with a python program, let's call it "stage 1", which uses subprocess to call bash to source the variable file, as my above answer does, but it then tells bash to export all of the variables, and then exec the rest of your python program, which is in "stage 2".

Stage 1 python program:

#!/usr/bin/env python

import subprocess

status = subprocess.call(
  ['bash', '-c',
  '. myGlobalVariables.bash; export $(compgen -v); exec ./stage2.py'
  ]);

Stage 2 python program:

#!/usr/bin/env python
# anything you want! for example,
import os
for key in os.environ:
  print key, " = ", os.environ[key]
Community
  • 1
  • 1
Jeff Schaller
  • 2,352
  • 5
  • 23
  • 38
0

As stated in @theorifice answer, the trick here may be that such formatted file may be interpreted by both as bash and as python code. But his answer is outdated. imp module is deprecated in favour of importlib.

As your file has extension other than ".py", you can use the following approach:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader 

spec = spec_from_loader("foobar", SourceFileLoader("foobar", "myGlobalVariables.bash"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)

I do not completely understand how this code works (where there are these foobar parameters), however, it worked for me. Found it here.

Ashark
  • 643
  • 7
  • 16