0

I have seen many examples about this topic but i cannot find any suitable answer for me.I need your help to know the trick about this. Suppose, i have two function funA() and funB(). They take exactly same arguments. And the functions return two separate outputs i.e a pandas series object. I want to execute the functions at the same time without GIL. Here is my code sample:

from multiprocessing import Process
from queue import Queue
q1 = Queue()
q2 = Queue()
res1 = Process(target=funA,args=(a,b,c,q1))
res1.start()
res2 = Process(target=funB,args=(a,b,c,q2))
res2.start()
res1.join()
res2.join()
result1 = q1.get()
result2 = q2.get()

The aforesaid code gives the following traceback:

File "C:\Python34\lib\multiprocessing\reduction.py", line 59, in dump
 ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

I have gone through the following threads like http://sebastianraschka.com/Articles/2014_multiprocessing_intro.html, Concurrently run two functions that take parameters and return lists?, Make 2 functions run at the same time

Please help me. Thanking you in advance.

Community
  • 1
  • 1
sayak_ghosh90
  • 41
  • 10

1 Answers1

0

Use multiprocessing.Queue instead of queue.Queue: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • its giving the error: `_pickle.PicklingError: Can't pickle : attribute lookup module on builtins failed` @John – sayak_ghosh90 Nov 13 '14 at 10:38
  • @sayak_SIBIA: please post a complete runnable program then, because I copied your code and added the necessary pieces and it worked for me on both Python 2 and Python 3. – John Zwinck Nov 13 '14 at 12:43