run 2 or more subprocess call separately

Question

I have two different executable (run1 and run2). I want to run them simultaneously. They share one readonly input file, but otherwise, they are different process. Is it possible to call the call's simultaneously? Currently, as I have coded, cmd2's call wait for cmd1's call to complete.

with open(dosout, "w") as dout, open(jijout, "w") as jout:
    cmd1 = ["mpirun", "-np", "8", "~/WORK/run1", dosinp]
    cmd2 = ["mpirun", "-np", "8", "~/WORK/run2", jijinp]
    call(cmd1, stdout=dout)
    call(cmd2, stdout=jout)
dout.close()
jout.close()

Is it possible with call? or Popen from this answer is the only (or better) way out?

call waits for the process to return, Popen does not so yes you can use Popen — Padraic Cunningham, Sep 26 '15 at 13:20
http://stackoverflow.com/questions/16071866/non-blocking-subprocess-call — Padraic Cunningham, Sep 26 '15 at 13:24
related: [Python threading multiple bash subprocesses?](http://stackoverflow.com/q/14533458/4279) — jfs, Sep 26 '15 at 21:40
unlrelated: `subprocess` does **not** use the shell unless you ask; do you need to call `os.path.expanduser("~/WORK/run2")` here? — jfs, Sep 26 '15 at 21:48

score 1 · Answer 1 · answered Sep 26 '15 at 12:56

1

You'll need to use Popen as described in the answer you linked (and quite a few other places as well, because Popen is a slightly lower-level interface which is often needed).

answered Sep 26 '15 at 12:56

John Zwinck

239,568
38
324
436

jfs · Answer 2 · 2015-09-26T21:46:06.553

To run two subprocesses concurrently:

from subprocess import Popen

# start processes
with open(dosout, "wb", 0) as dout, open(jijout, "wb", 0) as jout:
    processes = [Popen(["mpirun", "-np", "8", "...WORK/run1", cmd], stdout=file)
                 for cmd, file in [(dosinp, dout), (jijinp, jout)]]

# wait for them to finish
statuses = [p.wait() for p in processes]

Note: the file are closed on exit from the with-statement, don't call .close() manually.

run 2 or more subprocess call separately

2 Answers2