Run multiple python file concurrently

Question

how to run multiple files of python simultaneously

I have three files pop.py pop1.py pop2.py i want to run this file concurrently this files are getting run one by one python code to run all files

You need to start different processes (threads aren't enough) so the easiest way is to start them manually. — sehigle, Dec 20 '18 at 09:18
Welcome to Stackoverfow. You are much more likely to get relevant answers if you can include some code in the question along with a description of what you expect to happen and what actually happens. This site is mostly about helping people to get their existing code working. even if it's a shell script - just let us see what you tried. — holdenweb, Dec 20 '18 at 09:31

score 7 · Answer 1 · answered Dec 20 '18 at 09:42

7

You can easily accomplish this with the subprocess module.

import subprocess

process1 = subprocess.Popen(["python", "pop.py"]) # Create and launch process pop.py using python interpreter
process2 = subprocess.Popen(["python", "pop1.py"])
process3 = subprocess.Popen(["python", "pop2.py"])

process1.wait() # Wait for process1 to finish (basically wait for script to finish)
process2.wait()
process3.wait()

answered Dec 20 '18 at 09:42

I want to run each file simultaneously – itsvinit Dec 21 '18 at 06:23
this runs all of them simultaneously, the only delay between the start times of the processes is the execution of the Popen function which is insignificant – Dec 22 '18 at 11:02

score 1 · Answer 2 · answered Dec 20 '18 at 09:54

Does it have to be a python solution? With the problem as stated, it might be easiest to just start all three in bash:

python pop.py &
python pop1.py &
python pop2.py &
wait # wait for all three to finish, if needed

While this solution runs them concurrently, you should think about why you want them to be concurrent. Are you trying to parallelize your computation? Are the processes communicating (e.g. a Producer/Consumer pattern)? Are there any dependencies between them? In all but the simplest cases, you would usually be better served by bundling all three python modules together into a python package and adding some runner code which imports all three, starts each as a thread (see oren revenge's answer), and handles any inter-process communication.

score 1 · Answer 3 · answered Dec 20 '18 at 10:00

I would recommend to read about threading within Python. You should think about rearranging your code in one file.

PSEUDOCODE

import threading

class Pop(threading.Thread):
    def run(self):
        # Content from "pop.py"
        # Maybe some some changes are needed


class Pop1(threading.Thread):
    def run(self):
        # Content from "pop1.py"


# ...

pop = Pop()
pop1 = Pop1()
# ...

pop.start()
pop1.start()
# ...

score 1 · Answer 4 · answered Sep 06 '19 at 06:40

1

import test1,test2
from threading import Thread

Thread(target=test2.main).start()
Thread(target=test1.main).start()

This script runs test1.py and test2.py concurrently. Hope this helps.

answered Sep 06 '19 at 06:40

Ajinkya Bhanudas

11
3

score 0 · Answer 5 · answered Dec 20 '18 at 09:49

0

create a Shell file like this

 python pop.py
 python pop1.py
 python pop2.py

and run .sh file. .sh Run multiple file one by one

answered Dec 20 '18 at 09:49

Nikulsinh

16

1

As written, this would run the three files sequentially. – Quantum7 Dec 20 '18 at 09:55

score 0 · Accepted Answer · answered Dec 20 '18 at 11:38

0

Make main() function in every python file and then import all the files in your main file. Then call all main functions.

from . import pop
from . import pop1
# and so on

# and now call all main functions
pop.main()
pop1.main()
# and so on

answered Dec 20 '18 at 11:38

crodev

1,323
6
22
39

Run multiple python file concurrently

6 Answers6

Linked