0

Maybe it's a very simple question, but I'm new in concurrency. I want to do a python script to run foo.py 10 times simultaneously with a time limit of 60 sec before automatically abort. The script is a non deterministic algorithm, hence all executions takes different times and one will be finished before the others. Once the first ends, I would like to save the execution time, the output of the algorithm and after that kill the rest of the processes.

I have seen this question run multiple instances of python script simultaneously and it looks very similar, but how can I add time limit and the possibility of when the first one finishes the execution, kills the rest of processes?

Thank you in advance.

Biowav
  • 140
  • 7

1 Answers1

1

I'd suggest using the threading lib, because with it you can set threads to daemon threads so that if the main thread exits for whatever reason the other threads are killed. Here's a small example:

#Import the libs...
import threading, time

#Global variables... (List of results.)
results=[]


#The subprocess you want to run several times simultaneously...
def run():
 #We declare results as a global variable.
 global results
 #Do stuff...
 results.append("Hello World! These are my results!")


n=int(input("Welcome user, how much times should I execute run()? "))
#We run the thread n times.
for _ in range(n):
 #Define the thread.    
 t=threading.Thread(target=run)
 #Set the thread to daemon, this means that if the main process exits the threads will be killed.
 t.setDaemon(True)
 #Start the thread.
 t.start()

#Once the threads have started we can execute tha main code.
#We set a timer...
startTime=time.time()
while True:
 #If the timer reaches 60 s we exit from the program.
 if time.time()-startTime>=60:
  print("[ERROR] The script took too long to run!")
  exit()
 #Do stuff on your main thread, if the stuff is complete you can break from the while loop as well. 
 results.append("Main result.")
 break

#When we break from the while loop we print the output.
print("Here are the results: ")
for i in results:
 print(f"-{i}")

This example should solve your problem, but if you wanted to use blocking commands on the main thread the timer would fail, so you'd need to tweak this code a bit. If you wanted to do that move the code from the main thread's loop to a new function (for example def main(): and execute the rest of the threads from a primary thread on main. This example may help you:

def run():
 pass

#Secondary "main" thread.
def main():
 #Start the rest of the threads ( in this case I just start 1).
 localT=threading.Thread(target=run)
 localT.setDaemon(True)
 localT.start()
 #Do stuff.
 pass

#Actual main thread...
t=threading.Thread(target=main)
t.setDaemon(True)
t.start()
#Set up a timer and fetch the results you need with a global list or any other method...
pass

Now, you should avoid global variables at all costs as sometimes they may be a bit buggy, but for some reason the threading lib doesn't allow you to return values from threads, at least i don't know any methods. I think there are other multi-processing libs out there that do let you return values, but I don't know anything about them so I can't explain you anything. Anyways, I hope that this works for you.

-Update: Ok, I was busy writing the code and I didn't read the comments in the post, sorry. You can still use this method but instead of writing code inside the threads, execute another script. You could either import it as a module or actually run it as a script, here's a question that may help you with that: How to run one python file in another file?

Asriel
  • 182
  • 9
  • First of all, thank you for your very clear and elaborate answer, but I think it doesn't do exactly what I need. The threads executes a function. If one of the threads ends, that function generate an output, I need to catch that output. At this point, the rest of the threads doesn't interest to me so I would like to kill them – Biowav Jun 08 '20 at 13:01
  • Hi @Biowav , I get what you mean, but I think that it is possible to catch output from imported files, I'm not sure how, as I don't usually work with them, but I still think you should check it out, you might find something useful – Asriel Jun 08 '20 at 13:56
  • Have you tried looking at command pattern? The killing of threads would not be the right way, as at the hardware level the gates become loose and can damage the hardware – Shreyas Moolya Apr 30 '21 at 05:24