Python: Execute multiple Scripts simultaneously from same Interpreter

Question

I have a python script, which is executing again 4-5 python scripts. For performance reasons i want to use same interpreter for executing all the script.

How could I handle this issue?

Answer 1

The obvious solution (which may require a little tweaking) is to just call the main function of each script from a master script. E.g., if script1.py contains:

#!/usr/bin/python
def main():
  // Do something
if __name__ == "__main__":
   main()

put in master.py

#!/usr/bin/python
import script1
def main():
  script1.main()

if __name__ == "__main__":
  main()

You can continue this pattern for as many scripts as you want.

Answer 2

Maybe you're looking for the execfile function in Python 2.x.

In Python 3 it was removed, but there are simple alternatives.

Answer 3

I wrote a package to execute multiple scripts from the same interpreter (sequentially not simultaneously).

Installation:

pip install mand

---

Usage:

mand script1.py script2.py script3.py script4.py

You can specify module paths or module names.

---

You may be able to run scripts at the 'same time' on using the runpy stdlib module (for python3) and threading stdlib module. Where you call runpy.run_path or runpy.run_module in separate threads, but you will only see performance benefits if the modules are IO bound and not CPU bound.

---

Using multiprocessing or os.system will spawn separate interpreters for each script, so the modules wouldn't be running in the same interpreter.

Answer 4

The currently executing interpreter is available in sys.executable. You can just pass that explicitly to subprocess.Popen as the first argument, or pass it as the 'executable' argument.

Answer 5

I don't think this is recommended, but worst case scenario you could make the system 'run' each script from within another:

import os
os.system('python script1.py')
os.system('python script2.py')
os.system('python script3.py')
os.system('python script4.py')