0

I am completely new to python multiprocessing. I am trying to implement the process in my code. The code runs fine for a limited number of processes. But as I increase the number of processes, then the process goes into an infinite loop.

Basically, I am trying to parallelize the square function, and storing the value in the array. The functions return the values via queues.

from multiprocessing import Process,Queue
import time
import numpy as np
import os


def print_square(num,pos,a,que1):
    """
    function to print square of given num
    """
    a[pos] = num * num

    que1.put((pos,a))

    return




if __name__ == "__main__":

    # number of process
    NP=int(100)

    # creating thread
    p=[]
    num = 2

    a = np.zeros(NP)
    que1 = Queue()
    for i in range (NP):

        p1 = Process(target=print_square, args=(num,i,a,que1))
        # q.append(que1)
        num = num + 1
        p.append(p1)


    print('process created')

    start_time=time.time()
    # starting processes

    for process in p:
        print('cr - i', process)
        process.start()

    print('procesS started')

    for process in p:
        print('join',process,time.time()-start_time)
        process.join()
        process.terminate()

    result = []
    print('result started')
    for process in p:
        print(process)
        result.append(que1.get(block=False))
    print('result completed')
    # result.sort()

    result1 = [r[1] for r in result]

    for i in range(len(result1)):
        a[i] = result1[i][i]


    a1 = time.time()-start_time
Paul
  • 5,473
  • 1
  • 30
  • 37

1 Answers1

0

It's possible that you are hitting a machine/OS limitation. For example, see here. How many cores do you have on your machine? I doubt it is anywhere close to 100 and therefore it is unlikely that you would need to spawn off that many processes.

Paul
  • 5,473
  • 1
  • 30
  • 37
  • I have 24 processors. I need to spawn that many processes. However, I figured out the problem. The actual problem was: I was joining process before getting the queue, this was causing the problem and putting it into the deadlock. I appreciate your response. Thank you..!! – Patel Parth Rajendrabhai May 30 '18 at 23:23