0

How to run multiple python scripts from python/shell script in background (eq. run them immediatelly as if started from terminal), but wait for execution of the second one and restart first one, when second one ends?

Basically what I want to do (pseudocode)

start first.py
for x in range(0,10):
    start second.py (args =x, x+1, ...)
    wait_for_second_to_end()
    restart_first_one()

I guess it can be done with threading, where I call os.system() - wait for one thread to end, then kill another one. But i want those scripts to run with as many resources as possible, they both create multiple threads. Will using threads slow it down?

Stamatis Tiniakos
  • 698
  • 1
  • 11
  • 33
protonderee
  • 1
  • Please provide a [mre] -- real code, not pseudocode -- that shows that you've tried to read and follow existing answered Q&A on the topic (we have quite a lot!). Note that `os.system()` should generally be avoided -- it takes a lot of care to use it securely; `subprocess.run()` is less trouble-prone, or in a lot of cases you can just use `multiprocessing` instead of starting the subprocesses by hand. – Charles Duffy Apr 03 '22 at 18:25
  • ...yes, trying to create many more threads than you have CPUs can result in contention and your overall system getting slower than it would have otherwise been; this is part of what thread pools (or process) pools are for; these are likewise a feature `multiprocessing` supports. (That said, you should generally use `threading` instead of `multiprocessing` unless you know _for a fact_ that the GIL is preventing your script from taking advantage of all available resources; the overhead from passing data between processes is substantial, so `multiprocessing` often costs more than it's worth). – Charles Duffy Apr 03 '22 at 18:28
  • I would argue this question is not a duplicate of https://stackoverflow.com/q/61843030/1364242 or is badly worded. It asks a direct quesion of not only how to start a script, but also how to stop and re-run it. i.e. relates to theading as much as runpy. I don't have reopen permissions but do have a concrete answer. – Jay M Apr 22 '22 at 11:30
  • See gist: https://gist.github.com/the-moog/415965a83074366d386db2baf5b6a371 – Jay M Apr 22 '22 at 12:06

0 Answers0