15

I am trying to create 86 instances of task.py to run simultaneously.

import sys
import subprocess

for file in range(86):
    subprocess.call([sys.executable,'task.py',str(file)+'in.csv',str(filen)+'out.csv'])
user14372
  • 1,409
  • 2
  • 11
  • 12

1 Answers1

22

subprocess.call waits for command to complete. Use subprocess.Popen instead:

import sys
import subprocess

procs = []
for i in range(86):
    proc = subprocess.Popen([sys.executable, 'task.py', '{}in.csv'.format(i), '{}out.csv'.format(i)])
    procs.append(proc)

for proc in procs:
    proc.wait()
Pe Dro
  • 2,651
  • 3
  • 24
  • 44
falsetru
  • 357,413
  • 63
  • 732
  • 636
  • Is this solution a "multithread" solution? Or actually it launches multiple instance of python? – Lisa Jul 07 '17 at 18:19
  • @Lisa, As the name of the module suggest, it launches multiple instances (sub"process"). (OP's original code also launches multiple instances) – falsetru Jul 08 '17 at 00:18
  • 1
    I was just searching for something like this for so much time. Thank you – Pawankumar Dubey Jan 04 '18 at 07:01
  • I can't get it to work when looping and appending: `out =subprocess.Popen("hugo server -D", stdout = subprocess.PIPE)` `outputs.append(out)` And then iterating through outputs and calling `output.wait()`. It opens up the first program, but then doesn't open up the next one unless I close the first program. – MasayoMusic Aug 13 '19 at 22:16
  • 2
    @MasayoMusic, Replace `"hugo server -D"` -> `['hugo', 'server', '-D']`. If that does not work, please post a separate question so that others can answer you. – falsetru Aug 13 '19 at 22:40
  • @falsetru I did create one yesterday (just updated), but I couldn't get an answer so I have been hunting around: `https://stackoverflow.com/questions/57466534/cant-launch-multiple-local-servers-using-subprocess-in-python` – MasayoMusic Aug 14 '19 at 00:32
  • @falsetru What is the purpose of the `sys.executable` in the Popen command...I looked into the official docs but could not find them using it. – Pe Dro May 29 '20 at 02:36
  • @PeDro, [`sys.executable`](https://docs.python.org/3/library/sys.html#sys.executable) is an absolute path to the executable of current python process. It was coming from OP's code. My assumption is that OP wanted to use the same python executable for the sub-process. – falsetru May 29 '20 at 04:55