2

The goal of this program is to calculate the loop_random for the loop num . I want to calculate 4 random different num/4 with 4 processors in parallel and in the end summing to give the u_total. But the problem is though I see all the 4 processors in action, the end result is calculated for 4 times the same set of num/4. But I want to calculate loop_random for 4 different random arrays. Here is the snippet of the part of the code I am working on. . 

from numpy import *
import multiprocessing as mp
from multiprocessing import Process, Queue
num = 5000
def loop_random(num, out):
    n = 1
    total = zeros(((100.,100.,100.)), dtype='float')
    while n<= num:
        x, y , z = random.rand(100), random.rand(100), random.rand(100) #some random numbers
        result = x**2 + y**2 + z**2
        total = total + result
        n += 1
    out.put(total)

if __name__=='__main__':
    q1 = mp.Queue()
    q2 = mp.Queue()
    q3 = mp.Queue()
    q4 = mp.Queue()
    p1 = mp.Process(target = loop_random, args=(num/4, q1))
    p2 = mp.Process(target = loop_random, args=(num/4, q2))
    p3 = mp.Process(target = loop_random, args=(num/4, q3))
    p4 = mp.Process(target = loop_random, args=(num/4, q4))
    p1.start()
    p2.start()
    p3.start()
    p4.start()
    u_total_proc1 = q1.get()
    u_total_proc2 = q2.get()
    u_total_proc3 = q3.get()
    u_total_proc4 = q4.get()
    p1.join()
    p2.join()
    p3.join()
    p4.join()
    u_total = u_total_proc1 + u_total_proc2 + u_total_proc3 + u_total_proc4
    print u_total

Any suggestions are welcome. The problem is more about is this the right way of multiprocessing? Does each Queue calculate the same result and not stored or each processor calculates different results and stored? Thanks a lot for your help. Please do correct me if there is something wrong in what I said.

geekygeek
  • 323
  • 1
  • 3
  • 11
  • It might help if you posted a working example, there are lots of syntax errors that prevent your sample from running – bj0 Feb 04 '14 at 17:47
  • Oh, I am sorry. Since it was part of the program, I didn't have a look. I will edit it now. Thanks. – geekygeek Feb 04 '14 at 17:49
  • Now its working :) Please check – geekygeek Feb 04 '14 at 18:11
  • Can you provide more of an explanation or raw values for what your expected u_total is? From your description, it sounds like you're expecting (loop_random(x/4) * 4) = loop_random(x), which won't work with total getting initialized to zeros every time. – Dane White Feb 04 '14 at 19:49
  • Thank you. Yes, I am expecting `(loop_random(x/4) * 4) = loop_random(x)`. So you think it is happening because I am initializing the total to zero? I will initialize total outside the loop and try. – geekygeek Feb 04 '14 at 19:53
  • I'm having trouble figuring out what you're trying to get. Can you provide an example input/expected output? The way ```loop_random``` is written (the calculation inside the loop is identical for each iteration), it can be simplified down to ```loop_random(x): 14*x*arange(0,100,1)**2``` which I'm guessing isn't what you're looking for. – bj0 Feb 04 '14 at 20:02
  • note that ```loop_random(x/4)*4 = loop_random(x)``` *is* true with the currently posted version. – bj0 Feb 04 '14 at 20:07
  • @jayanthcv, nevermind about my comment regarding the initialization of total. I misread your calculation. – Dane White Feb 04 '14 at 20:10
  • Dear bj0 , sorry for not being clear. I want to calculate `loop_random` for 4 different random arrays. But the program calculates 4*the same array. Thanks – geekygeek Feb 05 '14 at 08:23
  • possible duplicate of [Using python multiprocessing with different random seed for each process](http://stackoverflow.com/questions/9209078/using-python-multiprocessing-with-different-random-seed-for-each-process) –  Sep 14 '15 at 15:46
  • Did you ever figure out your problem? – Charlie Parker Apr 05 '17 at 03:14

1 Answers1

1

Sorry for posting this answer, as it is not an answer, however I cannot post comments yet.

Just wanted to note that using:

from pylab import *

Is considered bad practise. Don't blanket import a bunch of random stuff. Each module should have a longish list of the specific things it needs.

According to the Python Zen:

Explicit is better than implicit.

Can't argue with that :)

cpb2
  • 798
  • 1
  • 9
  • 17