0

I want to run the same function (with some random numbers generated inside it) many times by parallelizing the runs over multiple CPUs. I tried using multiprocessing but it seems like the function is running only once. I wonder if some caching is going on that prevents multiple runs from happening. Consider the following simple example.

import numpy as np
import multiprocessing as mp

p = mp.Pool()
vals = p.map(np.random.normal, [0]*5)
p.close()
print(vals)

This gives the output:

[-2.5729019061921052, -2.5729019061921052, -2.5729019061921052, -2.5729019061921052, -2.5729019061921052]

I want the function to run for every element in the list passed to map. How to disable this "caching" or whatever is going on? This issue doesn't arise when I use the built-in map instead of multiprocessing.

I am using python 3.8.8 on WSL.

Akhil
  • 115
  • 7
  • 1
    I have no information how `np.random.normal` work but you looking for this https://stackoverflow.com/a/71073861/10556711. Also this https://stackoverflow.com/questions/29854398/seeding-random-number-generators-in-parallel-programs link has some answer – Veysel Olgun Aug 03 '22 at 00:26
  • 1
    Thanks! That seems to answer the question. So it's a ```numpy``` issue, not a ```multiprocessing``` issue. – Akhil Aug 03 '22 at 04:15

0 Answers0