0

I hope i can explain this well.. (english is not my first language) but in this question they ask to run multiple python scripts simultaneously. this is exactly how im doing it right now, basically just excecuting my multiple scripts with & in bash

what im trying to do is avoid creating multiple scripts and would like a script that can run all of them simultaneously

on of the scripts looks like this (similar to all my other scripts)

while True:
    text = "some shell script"
    os.system(text)

I finding it difficult to use a while loop, or any kind of loop because it executes them one after the other and it got really slow. I'm very unfamiliar with python and not so good at programming.. so any help would be great

skdadle
  • 155
  • 1
  • 2
  • 17
  • You are using too many pronouns. "he asks", "doing it", "multiple ones", "all of mine".. I am not in your head, I don't know what these words refer to. – Arne Aug 28 '18 at 07:50
  • I understand, tried to fix, hope its more understandable now :D – skdadle Aug 28 '18 at 07:55
  • What about `os.system(text &)` in a while loop ? (or `text = "some shell script &"` followed by `os.system(text)`) – Aserre Aug 28 '18 at 08:02
  • As far as I understand, you want to execute some script as often as possible, right? As a start, you should try to understand yourself what kind of problem you want to solve. For example, if you want to avoid waiting or if you need all of your cores to work on the problem. [This article here](https://medium.com/@nbosco/multithreading-vs-multiprocessing-in-python-c7dc88b50b5b) gives an introduction of the differences between threading and multiprocessing, that might help. – Arne Aug 28 '18 at 08:02
  • It is not clear what you are trying to do. Do you have multiple scripts (as multiple .py files) ? Are you running them from bash or another python script? The question you link is about running them from bash, your code snipplet is trying to run some subprcesses from python. You should post a minimal working example, otherwise we are just guessing what you are trying to do here. – jlanik Aug 28 '18 at 08:02
  • hi, thank you all for your comments, im trying to execute all of them simultaneously and as often as possible . and yes i have multiple .py files that i run from bash. I tried putting os.system(text) in a while loop, but that executed them one after the other and it was a lot slower than executing my scripts from bash like test01.py & test02.py & test03.py & .. thank you for the article im gonna read it now, hope i cleared up some confusion – skdadle Aug 28 '18 at 08:08

1 Answers1

1

You could use os.fork() to spawn in a new process for each script. i.e.

text = "some shell script"
pid = os.fork()
if pid == 0
    os.system(text)
    sys.exit()

This will make a new process and execute the script in it, then exit upon completion. Although doing so in a while loop would just keep creating new processes up until the os stops it.

If you have a list of programs you want to execute it would be better to iterate over them with a for loop. e.g.

programs = ['test1.py', 'test2.py', 'test3.py']
for program in programs:
    pid = os.fork()
    if pid == 0
        os.system(program)
        sys.exit()

I would also advice using subprocess.call() over os.system() as it was written to replace it. It allows you to easily handle the input and output of the program being executed.

jc1850
  • 1,101
  • 7
  • 16
  • hi thanks for answering, if i understood correctly that would be executing the exact same shell script over and over again. Is there a way to slightly manipulate that "some shell script" like, have some slight variation? – skdadle Aug 28 '18 at 08:12
  • thanks for the answer! but do you think there is a way to "merge" all of my scripts into one, instead of having many of them? like a while loop that can accommodate some changes to the script it needs to loop over with every iteration? once again, thanks a ton! – skdadle Aug 28 '18 at 08:24