Read variables from shell/bash script into python

Question

I need to run some Shell/Bash scripts inside Python (eg. using subprocess or system) and get the values of some variables inside those scripts. I know the name of the variables because they are the same in every one of then, but the values are different.

I cannot use regex to extract the variables content because sometimes those scripts come with processing, like if statements that will change the content of those variables based on the computer specs (eg. version of the linux kernel, architecture...).

But i will download those scripts, so i cannot simply add the variables to the env or echo the variables content from the script, i need to get the script variables from Python.

The really ideal is to avoid even save the script to the disk because it will ran several times and write/read from disk can be a problem. I'm using urllib to get the script from a web address.

What's the questions here? Do you need help on what? Please clear the exact question. — Ali Nikneshan, Dec 13 '15 at 19:47
How do i read Shell Script variables into Python from a script that the name is dynamic and cannot edit, just run or read it. — h0m3, Dec 13 '15 at 20:07
Do you know the variable name? Then you can run regex to find it. — Ali Nikneshan, Dec 13 '15 at 20:10
I know the name but they arent static text, in some cases they are processed by the script (so, i need to read the variables values at the end state of the script, to be more precise). — h0m3, Dec 13 '15 at 20:12
Maybe you can post an example? It is quite hard to understand what you want to read. — RedX, Dec 14 '15 at 09:19
Well, when you run a shell/bash script from python (eg. Using subprocess or system) you can get the script output and return code. I need to run a script and get the values of some variables inside those scripts. But i will download those scripts, so i cannot simply add the variables to the env or echo the variables content from the script, i need to get the script variables from the Python side. The really ideal is to avoid even save the script to the disk because it will ran several times and write/read from disk can be a problem. So i'm using urllib to get the script from a web address. — h0m3, Dec 14 '15 at 22:37

score 2 · Accepted Answer · edited May 23 '17 at 12:30

2

This answer seems to more or less cover what you're looking for. Why not just download the script, source it (though, the usual warnings associated with running unknown code apply in that scenario) and regex the output of "set" for the variables, given that you know their names?

#### testSource.sh file
foo="abcdef"
bar="uvwxyz"
if [[ -z $bar ]]; then
    var3="test"
else
    var3="testing"
fi

EDIT: The following example should eliminate the need to download the script to disk. This version actually downloads the above 'testSource' file and writes it directly to the stdin of a Bash sub-process -- you should be able to put the links you're trying to download into the 'urls' array and initialize the 'values' dictionary with keys corresponding to your desired variable names:

import subprocess
import re
import requests

urls = ['http://192.168.1.10/testSource.sh']

for url in urls:
    response = requests.get(url).content
    values = dict.fromkeys(['foo', 'bar', 'var3'])
    proc = subprocess.Popen(['bash', '0<'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True)
    proc.stdin.write(response)
    proc.stdin.flush()
    output, err = proc.communicate(input='set')
    for key in values:
        match = re.search('%s=(.*)$' % key, output, re.MULTILINE)
        if match: 
            values[key] = match.group(1)
            print('%s=%s' % (key,values[key]))

edited May 23 '17 at 12:30

Community

1
1

answered Dec 13 '15 at 21:07

rumdrums

1,322
2
11
25

Thanks, i'm avoiding downloading the file because it will do this several times (100+ and write/read from disk can be a problem). But it works better than any other solution that i can find. The scripts are checked by the repository maintainers in that case, dont need to worry about running unknown code. I know the name of the variables, the problem is that the maintainers dont care if theres some processing script to get them (like a variable has different values for different architectures). – h0m3 Dec 13 '15 at 21:21
@h0m3 I definitely see the urge to avoid downloading files. And while there's got to be a better way to do this, quickly playing around with a named pipes has given me an example I think you can taylor to avoid having to write it to a file at all.... mkfifo blah; cat testSource.sh > blah | bash 0< blah && (set -o posix; set) | grep foo I'm sure the Python subprocess module has got a very clean way to do this, but I'm not sure offhand. I think pushing the file content into a Pipe and then reading it when executing the source command is definitely the way to go. – rumdrums Dec 13 '15 at 21:31
I definitely agree with you, pushing the file content in a pipe is the best solution. I will try to come with something to be used with subprocess. – h0m3 Dec 13 '15 at 22:10
@h0m3 Check out the edit I made to my answer -- it avoids the unnecessary writes to disk. – rumdrums Dec 14 '15 at 08:07
WoW. Thats really awesome. I never thought of using Popen like that. It solved the problem, now i can just download the script and ran it. Thanks alot dude. – h0m3 Dec 14 '15 at 22:52
@h0m3 Good to hear. I learned a lot figuring that solution out -- you'll definitely want some error handling, with some try/expect and whatnot, but the basic structure of the above code snippet should be more or less good to go. – rumdrums Dec 15 '15 at 00:41

Read variables from shell/bash script into python

1 Answers1