0

When I use multiprocessing to create some processes, it seems that processes have the same state. How does it work?

I want each process has different state, so that the data created by numpy.random is indepenedent. One way is to use pid as seed, is there a more elegent approach?

from multiprocessing import Pool

import numpy as np
import os


def f(x):
    print(f'{os.getpid()}: {np.random.get_state()}')
    print(f'{os.getpid()}: {np.random.rand(5)}')


if __name__ == '__main__':
    with Pool(5) as p:
        p.map(f, [None] * 5)

吴慈霆
  • 523
  • 2
  • 15
  • Multiprocessing *fork* the initial process so all derived processes inherit from the same state, you can set a new state for each process using `set_state` but the best is to create a `np.random.RandomState` object as the doc of `seed` state. – Jérôme Richard Mar 27 '22 at 14:57

1 Answers1

0

Given that you are seeding from global state, you can consider using generator, that can be instantiated, rather that generating from global state.

Take a look into Generator that can be used for such purpose.

From example:

import numpy as np
rng = np.random.default_rng(12345)
rints = rng.integers(low=0, high=10, size=3)
print(rints)
-----
array([6, 2, 7])
Luka Rahne
  • 10,336
  • 3
  • 34
  • 56