Python subprocess module with pre-populated environment

Question

Question can be related to Use python subprocess module like a command line simulator

I have written some infrastructure code called my_shell to which you can pass shell commands of my application that looks like this

class ApplicationTestShell(object):
    def __init__(self):
        '''
        Constructor
        '''
        self.play_ground_dir = "/var/tmp/MyAppDir" 
        ensure_dir_exists_and_empty(self.play_ground_dir)

    def execute_command(self, command, on_success = None, on_failure = None):
        p = create_shell_process(self, self.play_ground_dir)
        sout, serr = p.communicate(input = command)
        if p.returncode == 0:
            on_success(sout)
        else:
            on_failure(serr)

    def create_shell_process(self, cwd):
        return Popen("/bin/bash", env= {WHAT DO I DO HERE?},cwd = test_dir, stdout=PIPE, stderr=PIPE, stdin=PIPE)

The interesting bit to me here is the env parameter. Python expects like a 'map' datastructure of all environment variable. My application requires several variables exported and set. The script for setting and exporting is generated by running say '/bin/appload myapp' (Assume appload is always available on the path). What I do currently is when I call p.communicate I do the following

p.communicate(input = "eval `/bin/appload myapp`;" + command)

So basically before running the command I call the infrastructure setup.

1. Is there any way to do this in a better fashion in Python. I somehow want to push the eval /bin/appload part to the env parameter on the Popen class OR as part of the shell creation process.
2. What are the problems with my current implementation? (I feel it is hacky but I may be wrong)

Answer 1

1. It depends on how /bin/appload myapp works. If it only guarantees that it will output bash syntax, then parsing that output in Python in order to construct the environment object there is almost certainly more trouble than it's worth (you might need to support parameter and variable expansion, subshells, process substitution, etc, etc). On the other hand, if you are sure that /bin/appload myapp will only ever output lines of the form "VARIABLENAME=someword", then that's pretty trivial to parse in Python and you could move it into your Python code if you like.

   There are an awful lot of different directions you could go with these requirements; you could capture the output of appload myapp into a tempfile and set the subprocess's $BASH_ENV to that filename; that would cause the shell to source your environment setup before running your command in a way that some might consider cleaner. You could give your command (with the eval-ing prefix) as the first argument to Popen and pass shell=True, and let Popen do the bash invocation on its own (setting $SHELL explicitly to bash if necessary). You could use bash's -c option to specify the code to run on the command line rather than via stdin. You could have a multi-tiered approach by invoking a shell from Python which eval's the appload myapp environment and then exec's another shell underneath it, so that the first doesn't show up in ps listings and the command given to create_shell_process has the shell all to itself (although that shouldn't really matter). You could do a lot of things, depending on what your concerns are with respect to how the shell is invoked, how it looks in ps listings, whether you want your command to still be run if the appload myapp output produces an error when eval'd, etc. But for a general solution, I think what you have is perfectly fine.

2. I don't see any real problems with the implementation, besides cosmetic things or minor things that probably only came from copying and pasting the code: create_shell_process doesn't use its cwd parameter, and the on_success and on_failure parameters look like they're optional but the defaults will break things (you can't call None).