I am a little bit confused about the multiprocessing.Pool functionality. I try to use it and are interested in catching a KeyboardInterrupt. The code is somewhat like this:
try:
pool = multiprocessing.Pool(max_processes, _init_proc, [arg1, arg2])
pool.apply(_generateTargetList)
except (KeyboardInterrupt, Exception) as err:
logger.error("Failure while generating:'%s'", err)
if pool:
pool.terminate()
pool.join()
The processes are working as expected, but catching the KeyboardInterrupt is not working. It just prints the following, while generating procs and procs and procs..
^CProcess PoolWorker-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-9:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 113, in worker
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
result = (True, func(*args, **kwds))
File "targets.py", line 307, in _generateTargetList
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
start, end = _adjustSize(start, end)
File "targets.py", line 258, in _adjustSize
pretargets_w[1])
File "targets.py", line 133, in __append
f.flush()
KeyboardInterrupt
racquire()
KeyboardInterrupt
Process PoolWorker-10:
Process PoolWorker-3:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
Process PoolWorker-7:
Process PoolWorker-2:
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
Does anyone have an idea what is wrong with the code?