Numpy memmap and multiple processes

Question

Can't understand the behaviour of threads in python. I was interesting about thread safety of np.memmap(), and wrote some example:

from multiprocessing import Process
import numpy as np
import time

mm = np.memmap("foo", dtype=int, mode = "w+", shape = (10,))

def foo(i):
    while True:
        mm[1]=np.random.randint(low=2, high=50)
        sleepfor = np.random.uniform(low=0.0, high=1.1, size=None)
        print "proces = ", i, " value = ", mm[1], " sleep = ", sleepfor

        time.sleep(sleepfor)

if __name__ == "__main__":
    processes = []
    p1 = Process(target=foo, args=(1,))
    p1.start() 

    p2 = Process(target=foo, args=(2,))
    p2.start()

    p3 = Process(target=foo, args=(3,))
    p3.start()

    p4 = Process(target=foo, args=(4,))
    p4.start()

It outputs this:

proces =  1  value =  13  sleep =  0.91652869794
proces =  2  value =  13  sleep =  0.91652869794
proces =  3  value =  13  sleep =  0.91652869794
proces =  4  value =  13  sleep =  0.91652869794
proces =  1  value =  6  sleep =  0.721062383299
proces =  2  value =  6  sleep =  0.721062383299
proces =  3  value =  6  sleep =  0.721062383299
proces =  4  value =  6  sleep =  0.721062383299
proces =  1  value =  24  sleep =  0.161040327399
proces =  2  value =  24  sleep =  0.161040327399
proces =  3  value =  24  sleep =  0.161040327399
proces =  4  value =  24  sleep =  0.161040327399
proces =  1  value =  22  sleep =  0.837121791675
proces =  2  value =  22  sleep =  0.837121791675
proces =  3  value =  22  sleep =  0.837121791675
proces =  4  value =  22  sleep =  0.837121791675
and so on...

Why random values are the same for different processes?

Why they change every 4 iterations?

And also how to write a test to check if numpy memmap is thread safe (is it?)

UPD:

Ok, random.seed() solves issue number one and number two? Then I commented sleep() to cause writing concurrency and also no errors occured. Does it mean that np.memmap() thread safe?

score 1 · Answer 1 · edited Sep 29 '15 at 10:25

1

I think this is a problem with np.random, although I do not know why. If you use mm[1] = random.randint(1, 20) instead it fixes the problem of the value being the same across all of the processes.

>>> proces =  1  value =  17  sleep =  0.0967840686219
>>> proces =  2  value =  13  sleep =  0.0967840686219
>>> proces =  3  value =  19  sleep =  0.0967840686219
>>> proces =  4  value =  14  sleep =  0.0967840686219

Using random.random also fixes the same problem for the sleep variable.

edited Sep 29 '15 at 10:25

Martin Evans

45,791
17
81
97

answered Sep 29 '15 at 09:04

kezzos

3,023
3
20
37

Sorry, why the down vote? – kezzos Sep 29 '15 at 09:09
I did not vote. Its not me. – user3003873 Sep 29 '15 at 09:10
So values are ok. But I tried to make sleep the same way . It is steel the same. – user3003873 Sep 29 '15 at 09:11
Thats because it is still using np.random. Try changing to random.random() or something similar – kezzos Sep 29 '15 at 09:12
This is a comment not an answer – Padraic Cunningham Sep 29 '15 at 09:12
Of course I changed to raandint - the same result – user3003873 Sep 29 '15 at 09:13

Numpy memmap and multiple processes

1 Answers1