10

I'm trying to automate a process using python. If I am just in the terminal the workflow looks like: 

user:> . /path/to/env1.sh
user:> python something.py
user:> exit
user:> . /path/to/env2.sh
user:> python something2.py
user:> exit

etc for a few more steps. Each env.sh spawns a new script with a whole slew of environment variables and whatnot set within the current directory. I'm pretty sure I need to use subprocess, but I'm not exactly sure how to go about it. Ideally the workflow would go: open new shell --> run some commands --> exit shell --> repeat as necessary.

EDIT: It seems some clarification is needed. I understand how to use subprocess.Popen() and subprocess.call() to call things from within the shell that the Python script was called from. This is not what I need. When one calls env.sh it sets a whole ton of environment variables and a few other pertinent things and then drops you into a shell to run commands. It is important to note env.sh does not terminate until one types exit after running desired commands. Using subprocess.call("./env.sh", shell = True) opens the shell and stops there. It is just like entering the command ./env.sh except that when one issues the exit command, the rest of the python script. So:

subprocess.call(". /path/to/env.sh", shell = True)
subprocess.call("python something.py", shell = True)

Does NOT do what I need it to do, nor does:

p = subprocess.Popen(". /path/to/env.sh", shell = True)
subprocess.call("python something.py", shell = True)
p.kill()
jfs
  • 399,953
  • 195
  • 994
  • 1,670
user3377586
  • 103
  • 1
  • 1
  • 5
  • My solution was to use Popen.communcate() I did however have a bit of trouble getting my input to pass but I found the answer here: http://stackoverflow.com/questions/12965023/python-subprocess-popen-communicate-equivalent-to-popen-stdout-read one needs to import PIPE from subprocess to be able to pass strings to your subprocess. I'm not entire sure which answer to pick because none of them were entirely right, and many suggested modules that are slated for deprecation. – user3377586 Mar 04 '14 at 06:35
  • if you think you know the answer then post it is *as an answer* instead of a comment and accept it. – jfs Mar 05 '14 at 13:18

5 Answers5

8

You can use subprocess:

>>> import subprocess
>>> subprocess.call('python something.py', shell = True)

Or you can use os:

>>> import os
>>> os.system('python something.py')

Here is an example (turn on your speakers):

>>> import os
>>> os.system('say Hello')
A.J. Uppal
  • 19,117
  • 6
  • 45
  • 76
  • 1
    I appreciate this answer but in my question I stipulate that the commands need to be run within the shell opened by /path/to/env.sh not just the shell from which I call the initial python script. – user3377586 Mar 04 '14 at 05:47
8

As I understand you want to run a command and then pass it other commands:

from subprocess import Popen, PIPE

p = Popen("/path/to/env.sh", stdin=PIPE)   # set environment, start new shell
p.communicate("python something.py\nexit") # pass commands to the opened shell
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • But that doesn't do the same thing that OP's code is doing now. `. whatever` is the same as `source whatever` which basically takes `whatever` and runs it in the current environment. You can think of it like C's `#include`. OP is using that to set up environment variables so that when the python program is executed, it has the correct environment. Your solution passes `python something.py\nnext` (as a string) to the script via stdin -- I'm not sure how that is the same at all... – mgilson Mar 05 '14 at 14:53
  • @mgilson: `source whatever` has **no** effect on the parent. It is **pointless** to call `call("source whatever", shell=True)` if your purpose is to change environment variables. – jfs Mar 05 '14 at 14:55
  • Sure it effects the parent. That's why you use it. That's why you do `source ~/.bashrc` after you change something. You pick up those changes in your current environment. FWIW, I just did a simple test equivalent to: `echo "export FOO=Bar" > foo.sh; . foo.sh; echo ${FOO}` and the second `echo` worked. Are you trying to say that this change doesn't propagate into python? I agree with you there, but that's not the question. We're trying to reproduce OP's *bash* code in python -- and python has a different mechanism for setting the environment in subprocess (as I've already pointed out). – mgilson Mar 05 '14 at 15:00
  • @mgilson: See [Calling the “source” command from subprocess.Popen](http://stackoverflow.com/q/7040592/4279). Also [read OPs comment](http://stackoverflow.com/questions/22163422/using-python-to-open-a-shell-environment-run-a-command-and-exit-environment/22198788?noredirect=1#comment33637856_22163422) – jfs Mar 05 '14 at 15:01
  • I think we're in agreement about `source`, but not OP's comment. I've of the opinion that OP just *thinks* he/she solved the problem using pipes (but really didn't) ... Or changed `env.sh` to evaluate input read from stdin (Yikes!). – mgilson Mar 05 '14 at 15:04
  • @mgilson: no. I'm afraid not. *"Sure it affects the parent. "* is incorrect. `source` does not create a child shell to run `foo.sh`, there is **no parent** in your example, `foo.sh` runs in **the same** shell (the same process). As I undestand, OPs `env.sh` script, in addition to exporting envvars, starts a new shell: imagine `exec tcsh` at the end. `.communicate()` is used to pass commands to *that* shell (`tcsh` in the example). – jfs Mar 05 '14 at 15:12
  • "There is no parent in your example" -- Ok, fair enough. yes, `source` runs in the same shell because it's a shell builtin. I suppose I was a little loose in my terminology. Anyway, if OP is `exec`ing the stuff piped in, then this is a correct answer. However, *if* that is the case, I would definitely encourage OP to rethink the approach... exec on stdin seems like a bad idea -- especially when so many better alternatives exist... (like passing parameters on the command line) – mgilson Mar 05 '14 at 16:15
6

subprocess calls (particular Popen) accepts an env argument which is a mapping of environement variables to values. You can use that. e.g.

env = {'FOO': 'Bar', 'HOME': '/path/to/home'}
process = subprocess.Popen(['python', 'something.py'], env=env)

Of course, usually, it's better to just call some functions after *import*ing something.py instead of spawning a whole new process.

mgilson
  • 300,191
  • 65
  • 633
  • 696
  • I didn't end up using env but you motivated me to poke around the docs one more time and I thoroughly read about the the Popen constructor and found what I needed. – user3377586 Mar 04 '14 at 18:37
  • It seems all OP wants is to [pass commands to the shell started by `env.sh`](http://stackoverflow.com/a/22198788/4279) – jfs Mar 05 '14 at 13:17
0

Here is a help for you.... For running command you could do:

1)

from subprocess import call
call(["ls", "-l"])

2)

import os
os.system("command")

Example:

import os
f = os.popen('date')
now = f.read()
print "Today is ", now

For enabling terminal you can import os module:

import os
os.system('python script.py')

Or as mentioned you can use import subprocess

MLSC
  • 5,872
  • 8
  • 55
  • 89
  • The commands I run need to be within the shell opened by /path/to/env.sh not the same shell the parent script is called from. – user3377586 Mar 04 '14 at 05:59
  • Ok read this [link](http://docs.python.org/2/library/subprocess.html#subprocess.Popen) – MLSC Mar 04 '14 at 06:11
  • Read these links: [1](http://docs.python.org/2/library/subprocess.html) [2](http://docs.python.org/2/tutorial/interpreter.html) [3](http://www.tutorialspoint.com/python/python_environment.htm) – MLSC Mar 04 '14 at 06:14
  • Please do not recommend [os.popen](http://docs.python.org/2/library/os.html#os.popen). It's "Deprecated since version 2.6: This function is obsolete." – anishsane Mar 04 '14 at 06:30
-2
p = subprocess.Popen(". /path/to/env.sh", shell = True, stdout=subprocess.PIPE).communicate()
subprocess.call("python something.py", shell = True).communicate()
user3273866
  • 594
  • 4
  • 8
  • This won't work. The idea of the first command is to set the environment for the second. You're setting the environment in the first command, but it won't propagate back to python and so when you execute the second command it still executes in the same environment as python. – mgilson Mar 04 '14 at 16:27
  • -1: `call()` returns an integer (exit code). It has no `.communicate()` method. – jfs Mar 05 '14 at 13:15