how can i call a def to be executed simultaneously?

Question

I want to call a function that takes a long time to execute(variable, from 0 to 300 sec.), it has to be executed as many times as there are lists in a json. How can I do it so that it executes three in parallel, for example?

To make it simpler the code in the explanation would be to call a function, this...

def f(x, j):
    print(str(x*x)+" "+" "+str(j))
    time.sleep(2)

j = 0
for i in range(10):
    j += 1
    f(i,j)

result

thread 1 : 0, 1
thread 2 : 1, 2
thread 3 : 4, 3
thread 1 : 9, 4

thread 2 : 16, 5
thread 3 : 25, 6
thread 1 : 36, 7
thread 2:  49, 8
thread 3:  64, 9
thread 1:  81, 10

Thread three could end before thread one, but I always want three runs.

check semaphore: https://superfastpython.com/thread-semaphore/ — ahmedakef, Feb 02 '23 at 17:08
https://stackoverflow.com/questions/2562757/is-there-a-multithreaded-map-function — Nick ODell, Feb 02 '23 at 17:09
Take a look at `ProcessPoolExecutor` and `ThreadPoolExecutor`. — Louis Lac, Feb 03 '23 at 22:55

score 1 · Answer 1 · answered Feb 03 '23 at 23:01

Take a look at ProcessPoolExecutor and ThreadPoolExecutor, which are API's that allow the execution of concurrent tasks:

from concurrent.futures import ProcessPoolExecutor


def f(x, j):
  print(str(x*x)+" "+" "+str(j))
  time.sleep(2)


j = 0
with ProcessPoolExecutor() as e:
  for i in range(10):
    e.submit(f, i, j)

You can also use the .map() method in this case. Read the documentation for more information.

how can i call a def to be executed simultaneously?

1 Answers1