translating bash script to python

Question

I have some simple bash scripts that I would like to translate in python. I need to define some bash environment variables before calling the external program with subprocess. In particular I need to load a module environment. The original bash looks something like this

#!/bin/bash 
source /etc/profile.d/modules.sh
module purge
module load intel

./my_code

I want to call `my_code` with subprocess, but how do I pass the variables defined by `module` to the shell?

Edit:

my first attempt to convert the above script is

#!/usr/bin/python -tt
import subprocess

subprocess.call("source /etc/profile.d/modules.sh",shell=True)
subprocess.call("module purge",shell=True)
subprocess.call("module load intel",shell=True)
subprocess.call("./mycode",shell=True)

but this fails, because the environment variables (that I think are LIBRARY_PATH and PATH and C_INCLUDE) are modified in the shell launched by subprocess, but then this shell dies and these variables are not inherited by subsequent shells. So the question is: how can I launch a modules command and then save the relevant variables with os.environ ?

I think now the question is clear, isn't it? – simona Mar 19 '14 at 14:32 — simona, Mar 19 '14 at 14:32

score 1 · Accepted Answer · edited May 23 '17 at 12:28

1

You could use a multiline string so that the environment is modified in the same shell process:

from subprocess import check_call

check_call("""
source /etc/profile.d/modules.sh
module purge
module load intel

./my_code
""", shell=True, executable="/bin/bash")

See also Calling the “source” command from subprocess.Popen

edited May 23 '17 at 12:28

Community

1
1

answered Mar 20 '14 at 15:31

jfs

399,953
195
994
1,670

yeah, this is certainly a solution. but you know the above code is a simplification, between module load and the call to mycode I do some string manipulation and I actually call several programs that use the same modules, so this would not be very elegant – simona Mar 20 '14 at 15:33
@simona: what do you mean: *"would not be very elegant"*? – jfs Mar 20 '14 at 15:35
it seemed cleaner to me to pass the environment variables from one shell to the other, like you would do in bash with export, there must be a way to do that. – simona Mar 20 '14 at 15:38
@simona: Why do you need several shell processes? Also, check out [the link that I've added above](http://stackoverflow.com/a/22086176/4279) – jfs Mar 20 '14 at 15:39

score 0 · Answer 2 · answered Mar 19 '14 at 14:45

0

import os

os.environ is a dict of environment variables (exported shell variables)

answered Mar 19 '14 at 14:45

scav

1,095
1
9
17

I modified my question – simona Mar 20 '14 at 15:27

translating bash script to python

I want to call my_code with subprocess, but how do I pass the variables defined by module to the shell?

2 Answers2

I want to call `my_code` with subprocess, but how do I pass the variables defined by `module` to the shell?