4

I am a little bit confused about the multiprocessing.Pool functionality. I try to use it and are interested in catching a KeyboardInterrupt. The code is somewhat like this:

try:
    pool = multiprocessing.Pool(max_processes, _init_proc, [arg1, arg2])
    pool.apply(_generateTargetList)
except (KeyboardInterrupt, Exception) as err:
    logger.error("Failure while generating:'%s'", err)
    if pool:
        pool.terminate()
        pool.join()

The processes are working as expected, but catching the KeyboardInterrupt is not working. It just prints the following, while generating procs and procs and procs..

^CProcess PoolWorker-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-9:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 113, in worker
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    result = (True, func(*args, **kwds))
  File "targets.py", line 307, in _generateTargetList
    task = get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
    start, end = _adjustSize(start, end)
  File "targets.py", line 258, in _adjustSize
    pretargets_w[1])
  File "targets.py", line 133, in __append
    f.flush()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
Process PoolWorker-10:
Process PoolWorker-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
Process PoolWorker-7:
Process PoolWorker-2:
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):

Does anyone have an idea what is wrong with the code?

Reblochon Masque
  • 35,405
  • 10
  • 55
  • 80
mkind
  • 2,015
  • 2
  • 20
  • 25
  • 3
    look at this blog post, maybe it can help you : http://www.bryceboe.com/2010/08/26/python-multiprocessing-and-keyboardinterrupt/ – lucasg Oct 30 '13 at 20:56

0 Answers0