I wrote trivial Python multiprocessing to test how it works: Multiprocessing only works for the first iteration .I wrapped some array inside a function, then I called multiprocessing map to process it using 5 processor. All that was fine except the output which is 5 copies of each array. Any idea of how to return one instance of each array instead of five instances. I wondering if there is a general way to make the processes communicate with each other.
import numpy as np
import multiprocessing as mp
B=np.full((5,4,4),np.nan)
C=np.full((5,4,4),np.nan)
D=np.full((5,4,4),np.nan)
def job1(i):
print(i)
A=np.ones((4,4))
B[i,:,:]=A+1
C[i,:,:]=2*A+2
D[i,:,:]=A+5
return B,C,D
#%%
P=mp.Pool(5)
result=P.map(job1,np.arange(5))
P.close()
P.join()
result[0]
(array([[[ 2., 2., 2., 2.],
[ 2., 2., 2., 2.],
[ 2., 2., 2., 2.],
[ 2., 2., 2., 2.]],
[[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan]],
[[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan]],
[[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan]],
[[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan],
[nan, nan, nan, nan]]]),