0

I am trying to write a simple program using python's pool module. Here is my code:

from multiprocessing import Pool
from multiprocessing.pool import ApplyResult
import time
import os

def unwrap_self_f(arg, **kwarg):
  print 'inside unwrap_self_f'
  return C().f(arg, **kwarg)

class C:
  def f(self, name):
    return 'Inside f..Process id :' + str(os.getpid())

  def run(self):
    print 'Reached inside run..'
    print 'main process id ..', os.getpid()
    pool = Pool(processes=4)
    names = ['frank', 'justin', 'osi', 'thomas']        
    print pool.map(unwrap_self_f, names, 1)               

if __name__ == '__main__':
   print 'Starting....'
   c = C()
   print 'Running....'
   c.run()

When I run this program, I was expecting to see 4 different process ids being printed, one for each child process that was created. The chunksize that I have selected is 1, so that each item in the list goes to a separate process.

However the process id being printed is the same. I get the below output:

Starting....
Running....
Reached inside run..
main process id .. 1284
['Inside f..Process id :6872', 'Inside f..Process id :6872', 'Inside f..Process id :6872', 'Inside f..Process  id :6872']

Am I getting somewhere wrong here. Please can anyone shed some light on this. Thanks in advance.

LearnToLive
  • 462
  • 9
  • 25
  • It works for me under OS X. So, what OS are you using, and how many cores does your machine have? – deets Nov 23 '14 at 21:14
  • For me it works *sometimes* (ubuntu 12.04.5). After adding `time.sleep(1)` in the worker function, I get different ids. – hitzg Nov 23 '14 at 21:16
  • https://docs.python.org/2/library/multiprocessing.html#multiprocessing.pool.multiprocessing.Pool.map Ref the docs, it says "The (approximate) size of these chunks can be specified by setting chunksize to a positive integer." I guess there's no guarantee that specifying the chunksize will work - what happens if you don't specify it at all? – Tom Dalton Nov 23 '14 at 22:38
  • Also see http://stackoverflow.com/questions/3822512/chunksize-parameter-in-pythons-multiprocessing-pool-map and http://stackoverflow.com/questions/13264435/do-multiprocessing-pools-give-every-process-the-same-number-of-tasks-or-are-the – Tom Dalton Nov 23 '14 at 22:40
  • Thanks to all for their help. Following @hitzg comment. I added some time delay in the f method and increased the passed list. I was able to see different ids. I think python decides to handover tasks to different processes on their availability. – LearnToLive Nov 24 '14 at 11:53

0 Answers0