Retain environment of helper python script in main script

Question

I've one helper script which I want to call from main script which is acting as a Server. This main script looks like this:

Class Stuff():
    def __init__(self, f):
        self.f = f
        self.log = {}

    def execute(self, filename):
        execfile(filename)

if __name__ == '__main__':
    #start this script as server
    clazz = Stuff()
    #here helper_script name will be provided by client at runtime
    clazz.execute(helper_script)

Now Client will invoke this helper script by providing it's name to main script(Server). After execution I want to retain variables of helper script (i.e: a,b) in main script. I know one way is to return those variables from helper script to main script. But is there any other way so to retain all variables of helper script. This is how helper script looks like:

import os
a = 3
b = 4

I tried using execfile and subprocess.

Answer 1

You can pass a locals dictionary to execfile. After executing the file, this dictionary will contain the local variables it defined.

class Stuff():
    def __init__(self):
        self.log = {}

    def execute(self, filename):
        client_locals = {}
        execfile(filename, globals(), client_locals)
        return client_locals

if __name__ == '__main__':
    #start this script as server
    clazz = Stuff()
    #here helper_script name will be provided by client at runtime
    client_locals = clazz.execute('client.py')
    print(client_locals)

With your client.py, this will print:

{'a': 3, 'b': 4, 'os': <module 'os' from '/Users/.../.pyenv/versions/2.7.6/lib/python2.7/os.pyc'>}

A word of warning on execfile: don't use it with files from untrusted sources.

Answer 2

As mentioned in the comments, you should return the value(s) from the helper script. If there is more than one, stick it in a list or dictionary (namedtuple is handy for this).

The key is that you need to assign the result of execfile to a variable in the server script. This doesn't have to be anything major, it can just be dumped straight into a collection such as a list.

Return-based code

Main file

class Stuff():
    def __init__(self, f):
        self.f = f
        self.log = {}
        self.storage = []

    def execute(self, filename):
        self.storage.append(execfile(filename))

if __name__ == '__main__':
    #start this script as server
    clazz = Stuff()
    #here helper_script name will be provided by client at runtime
    clazz.execute(helper_script)

Helper file (helper.py)

a = 3
b = 4
return (a, b)

Stdout alternative

An alternative approach would be to have the helper file print the results but redirect stdout. This is fairly trivial if you are using subprocess and is well detailed in the documentation (I can add an example if it would be beneficial). I'm less familiar with the exec / execfile approach than subprocess but there is an explanation on this SO question.

In case anything happens to the other SO question, copied here is the code for the two examples:

Frédéric Hamidi

    code = """
i = [0,1,2]
for j in i :
print j
"""

from cStringIO import StringIO
old_stdout = sys.stdout
redirected_output = sys.stdout = StringIO()
exec(code)
sys.stdout = old_stdout

print redirected_output.getvalue()

Jochen Ritzel

import sys
import StringIO
import contextlib

@contextlib.contextmanager
def stdoutIO(stdout=None):
    old = sys.stdout
    if stdout is None:
        stdout = StringIO.StringIO()
    sys.stdout = stdout
    yield stdout
    sys.stdout = old

code = """
i = [0,1,2]
for j in i :
    print j
"""
with stdoutIO() as s:
    exec code

print "out:", s.getvalue()

Answer 3

@Daniel Hepper Thanks for the clean explanation.In the case of Python3.x the execfile is not supported. you can refer to https://stackoverflow.com/a/437857/1790603

Answer 4

You can access all symbols (variables, classes, methods...) that you defined in any given namespace using the locals() method. Here's an example

import os
a = 1
b = 2
return locals()

will yield a dict containing (among some technical stuff)

{'os': <module 'os' from '...'>, 'a': 1, 'b': 2}

So, if saved the above in a file called foo.py, you could work with this value like this:

foo_vals = execfile(`foo.py`)
print(foo_vals["a"])
> 1

But that is not the approach I'd go with. Depending on the amount of code in your script, that dictionary is huge and cluttered with stuff you don't need, and I would only ever use it for debugging purposes. Here are other options, ordered by "good coding practice":

1. Put your code inside a class that you import and then initialize & run to your liking, keeping all relevant variables inside that class's namespace.
2. You could call import foo and then access a and b with foo.a, foo.b. (You can call import foo anywhere inside your code though I think it will only execute once).
3. Explicitly return the variables in your script that you need, then exec it with execfile