python debug tools for multiprocessing

Question

I have a python script that works with threads, processes, and connections to a database. When I run my script, python crashes.

I cannot explicitly detect the case in which this happens.

Now I am looking for tools to get more information when python crashes, or a viewer to see all my created processes/connections.

I've just started working with threads and I've ended up using logging.debug("I made it to function X") everywhere. — Paul Collingwood, Nov 23 '12 at 21:13
Are you on Windows? If so, have you tried building and running a debug build of Python, then capturing the callstack when it crashes by attaching Visual Studio (after it crashes) and "breaking" (with the pause button) into the crash location? EDIT: To be clear, your script causes Python itself (not your script) to crash on occasion, right? — Cameron, Nov 23 '12 at 21:15
there is faulthandler module. There are issues with [mixing threading and multiprocessing without proper care](http://stackoverflow.com/questions/12984003/status-of-mixing-multiprocessing-and-threading-in-python) — jfs, Nov 23 '12 at 22:33

User · Answer 1 · 2013-06-12T09:58:25.580

I created a module RemoteException.py that shows the full traceback of a exception in a process. Python2. Download it and add this to your code:

import RemoteException

@RemoteException.showError
def go():
    raise Exception('Error!')

if __name__ == '__main__':
    import multiprocessing
    p = multiprocessing.Pool(processes = 1)
    r = p.apply(go) # full traceback is shown here

OLD ANSWER

I had the problem, too.

this is what i did... a RemoteException to debug multiprocessing calls

RemoteException.py

copy the source and remove line 19: file.write('\nin %s ' % (Process.thisProcess,)) and the line import Process

The problem is: multiprocessing only transfers the exception but looses the traceback. The code below creates an Exception object that saves the traceback. And prints it in the calling process.

In your script you can do something like this:

import RemoteException

def f():
    try:
        # here is code that fails but you do know not where
        pass
    except:
        ty, err, tb = RemoteException.exc_info() # like sys.exc_info but with better message
        raise ty, err, tb

# here follows your multiprocessing call to f

score 5 · Answer 2 · answered Jan 20 '18 at 05:03

5

pip install celery

then in your code:

from celery.contrib import rdb; rdb.set_trace()

this works like a remote pdb, like below:

Remote Debugger:6899: Ready to connect: telnet 127.0.0.1 6899

Type `exit` in session to continue.

Remote Debugger:6899: Waiting for client...

from another window telnet localhost 6899 and you have a full function pdb available.

answered Jan 20 '18 at 05:03

Bhakta Raghavan

664
6
16

This was extremely useful tip! The `pudb.remote.set_trace` would have been nicer, but it sadly does not support Windows. This worked like a charm! On Powershell, the command to connect is `Telnet 127.0.0.1 6899` (with default IP and port). I might extract the code from Celery as standalone packet to PyPi some day, if there is no alternative. – Niko Föhr Apr 21 '21 at 10:06
I noted that using backspace does not work (code looks okay but will give `SyntaxError` inside the debugger). Don't know if it's feature of telnet connections in general or just feature of the Powershell Telnet. – Niko Föhr Apr 21 '21 at 10:09

score 1 · Answer 3 · edited Nov 25 '12 at 20:30

1

I don't know your case, but if you use threads or multiprocessing then your code is applicable for parallel processing (usually). In difficult cases I do everything just calling a function without pool, catch error and then go to pools again.

edited Nov 25 '12 at 20:30

Mickaël Le Baillif

2,027
16
22

answered Nov 25 '12 at 19:58

crow16384

587
3
15

Without pool, can find some errors; but not able to find out interfacing between pooled tasks. Deadlock for example. – Herbert Yu Aug 23 '23 at 16:20

score 0 · Answer 4 · answered Nov 25 '12 at 20:06

There is a graphical debugger called WinPDB which gives you the option to follow a particular Python process. You can step through and see all the variables at different locations in the call stack.

It gives you the option of which process to follow at fork.

It will even capture the final exception and allow you to see where it originated.

http://winpdb.org/

python debug tools for multiprocessing

4 Answers4

Linked