1

Hi i am using python multiprocessing module and the problem is when i get an error from a code that has been executed in another thread, the Traceback is only pointing to multiprocessing/pool.py", line 520 instead of the line number of what caused the error exactly.

Traceback (most recent call last):
  File "analyzer.py", line 196, in <module>
    for ret in pool.imap(ProcessRow, rows):
  File "/usr/lib64/python2.6/multiprocessing/pool.py", line 520, in next
    raise value
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

so line 196 in analyzer.py is when i call for ret in pool.imap(ProcessRow, rows):

so it's completely not helpful....Any advice ?

Hakim
  • 1,242
  • 1
  • 10
  • 22

1 Answers1

2

The error is raised from ProcessRow() or some function called by it (why is your function CamelCaseUpper?).

You are not getting a proper stack trace, because the exception is actually pickled from the subprocess (hence the name of the module: multiprocessing) your code is executing in and is then raised again in the manager process. Hence the stack trace shows up as if it was local (while it is actually an exception raised in the subprocess).

The preferred way of dealing with this seems to be to catch the exception in the forked of code and printing a proper stacktrace from there.

dhke
  • 15,008
  • 2
  • 39
  • 56