1

So what I want to do is run the same function multiple times simultaneously while getting a result as return and storing it in an array or list. It goes like this:

def base_func(matrix,arg1,arg2):
    result = []
    for row in range(matrix.shape[0]):
        #perform necessary operation on row and return a certain value to store it into result
        x = func(matrix[row],arg1,arg2)
        result.append(x)
    return np.array(result)

I tried using threading in python. My implementation goes:

def base_func(matrix,arg1,arg2):
    result = []
    threads = []
    for row in range(matrix.shape[0]):
        t = threading.Thread(target=func,args=(matrix[row],arg1,arg2,))
        threads.append(t)
        t.start()
    for t in threads:
        res = t.join()
        result.append(res)
    return np.array(result)

This doesn't seem to work and just returns None from the threads.

user3666197
  • 1
  • 6
  • 50
  • 92
Subham Dutta
  • 11
  • 1
  • Does this answer your question? [Running same function for multiple files in parallel in python](https://stackoverflow.com/questions/25889268/running-same-function-for-multiple-files-in-parallel-in-python) – Querenker Dec 19 '20 at 19:52
  • The answer by @thepunitsingh seems to be correct. As just a suggestion, I think you can define `result` as a global variable and do appending into it in `func` function – Zar Ashouri Dec 19 '20 at 20:40

1 Answers1

1

From what I read in the documentation of threading.join(), it says:

As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

You will always get None from these line of your code:

res = t.join()
result.append(res)

This post mentions a similar problem as yours, please follow this for your solution. You might want to use concurrent.futures module as explained in this answer.

thepunitsingh
  • 713
  • 1
  • 12
  • 30