0

The following problem arised directly due to applying the answer to this question. In the minimal working example (MWE) there's a place in the myscript definition where I generate some random numbers, then perform some operations on them, and fnally write the output to a file. When this code is un-parallelized, it works correct. However, if it's parallel (I'm testing it on a 2-core machine, and have two threads at a time), when I want to perform 4 iterations (boot) I get twice the same output (i.e., among four outputs I get only two distinct numbers, not four as expected). How can this be fixed?

MWE:

import random
import math
import numpy as np
import multiprocessing as mp
from multiprocessing import Pool

boot = 4
RRpoints = 278

def myscript(iteration_number):
    RRfile_name = "outputRR%d.txt" % iteration_number
    with open(RRfile_name, "w") as RRf:

        col1 = np.random.uniform(0 , 1 , RRpoints)
        col2 = np.random.uniform(0 , 1 , RRpoints)
        sph1 = [i * 2 * math.pi for i in col1]
        sph2 = [math.asin(2 * i - 1) for i in col2]

        for k in xrange(0 , RRpoints):
            h = 0           
            mltp = sph1[k] * sph2[k]
            h += mltp

        RRf.write("%s\n" % h)

x = xrange(boot)
p = mp.Pool()
y = p.imap(myscript, x)
list(y)
Community
  • 1
  • 1
corey979
  • 141
  • 1
  • 9
  • 2
    That doesn't seem very minimal to me. Instead of doing arithmetic, you could simply draw a single random number; instead of writing to a file, you could simply use `print`; and you'd still see the problem. Then you could try using `random` instead of `np.random`. That would lead you to google `numpy random multiprocessing`, which might take you [here](http://stackoverflow.com/questions/14504866/python-multiprocessing-numpy-random) or [here](http://stackoverflow.com/questions/9209078/using-python-multiprocessing-with-different-random-seed-for-each-process). – DSM Aug 16 '15 at 13:41
  • @DSM Thanks to your links I found out that inserting `np.random.seed()` helps. If you make an answer I'd be happy to accept it. – corey979 Aug 16 '15 at 14:49
  • Is there no way to set the random number for every process that might use random numbers? Say one uses the module random, numpy, scipy, tensorflow and who knows what else. Is the only way to make sure the process has a different random seed to go through each of these and manually set the state? – Charlie Parker Apr 05 '17 at 03:15

0 Answers0