5

I have three .py files that I want to run at the same time in a Python script file. I initially called subprocess.call() three times (once for each .py file) but remembered that it blocks until the command is finished. I tried subprocess.Popen(['screen', 'python_file']) since I believe it doesn't block but when I was checking for the processes with screen -ls there was only one process running. How do I get all three programs running at the same time with a Python script? Should I be using the multiprocessing or multithreading library?

Edit: The other processes aren't supposed to finish since they're running in an infinite loop. Here's exactly what I had in my Python script file. I'm using screen because each .py file has stdout logged onto the terminal and I want to be able to see what is being logged for each one.


subprocess.Popen(['screen', './submitter.py'])
subprocess.Popen(['screen', './worker.py'])
subprocess.Popen(['screen', './tester.py'])

Petesta
  • 1,623
  • 3
  • 19
  • 29
  • 1
    `Popen` already does what you want. If you can see only one process then it simply means that the other processes already finished their work(or maybe they exited due to an error). – Bakuriu Aug 13 '13 at 18:21
  • It works for me. Can you show us an example? – tdelaney Aug 13 '13 at 18:22
  • Any reason why you are using 'screen'? Its not needed just to run the files. – tdelaney Aug 13 '13 at 18:30
  • @tdelaney An example of what is being displayed when I run `screen -ls`? – Petesta Aug 13 '13 at 18:37
  • What you've done works for me. Perhaps the .py files failed. You could create them all in a list, sleep a few seconds and then print out return codes `for p in procs:print p.returncode`. It'll be None if the proc is still running. – tdelaney Aug 13 '13 at 18:46
  • @tdelaney Thanks! It worked. It was just a bug on my part that I found. – Petesta Aug 13 '13 at 21:14

1 Answers1

9

If you want to use multiprocessing, you can try this:

import multiprocessing

def worker(file):
    # your subprocess code


if __name__ == '__main__':
    files = ["path/to/file1.py","path/to/file2.py","path/to/file3.py"]
    for i in files:
        p = multiprocessing.Process(target=worker, args=(i,))
        p.start()
Community
  • 1
  • 1
  • If a subprocess raises an uncaught exception, what will happen? – Jordan Warbelow-Feldstein Jul 22 '14 at 18:07
  • @JordanFeldstein I am not exactly sure what you mean by `uncaught exception`, but here is some documentation on exceptions in `subprocess`: https://docs.python.org/2/library/subprocess.html#exceptions –  Jul 23 '14 at 21:41
  • @enginefree That works! But I need those process come up with different CMD name by 'ps -ef'. Please kindly check up this question. http://stackoverflow.com/questions/43362708/run-multiple-python-scripts-concurrently-with-different-cmd-name – zzxwill Apr 12 '17 at 07:41