How to debug a python module that needs to be executed with -m?

Question

Every debugger I tried out there expects a source file to debug. However Python does not always work this way.

I have a module that is a folder with __init__.py and __main__.py files inside, among others, and I usually execute that this way:

$ cd /parent/folder
$ python3 -m module_folder --help

If I don't use -m, relative imports fail. If I just pass the folder to pudb, pdb and others, debugger fails:

$ cd /parent/folder
$ python3 -m pdb module_folder
Traceback (most recent call last):
  File "/usr/lib64/python3.3/pdb.py", line 1658, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib64/python3.3/pdb.py", line 1536, in _runscript
    with open(filename, "rb") as fp:
IsADirectoryError: [Errno 21] Is a directory: 'module_folder'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /usr/lib64/python3.3/pdb.py(1536)_runscript()
-> with open(filename, "rb") as fp:
(Pdb)

How can I debug this? Preferrably with pudb, and without removing the relative imports.

It would be nice if someone could find a way to achieve this that doesn't require modifying the program source and doesn't require changing the working directory. — Robie Basak, Dec 20 '16 at 16:39

Yajo · Accepted Answer · 2014-01-03T09:13:25.837

Put this at the top of your __main__.py:

#!/usr/bin/env python3

# Declare itself as package if needed
if __name__ == '__main__' and __package__ is None:
    import os, sys, importlib
    parent_dir = os.path.abspath(os.path.dirname(__file__))
    sys.path.append(os.path.dirname(parent_dir))
    __package__ = os.path.basename(parent_dir)
    importlib.import_module(__package__)

# Continue with your code
do_things()

This way, these 2 commands become equivalent:

cd /parent/folder; python -m module_folder
python /parent/folder/module_folder/__main__.py

Just use the 2nd syntax for debugging:

pudb /parent/folder/module_folder/__main__.py

or

python3 -m pdb /parent/folder/module_folder/__main__.py

The same works for Python 2.

You can safely remove from the above code the part __name__ == '__main__' and because, as you are writing in __main__.py, it would always be True. However, it's a common practice to put it. See PEP 366 and this other answer.

score 1 · Answer 2 · answered Dec 23 '13 at 13:07

1

I think you want to do this:

$ cd /parent/folder
$ python3 -m pdb -m module_folder

As it was, you were missing the second -m which was making Python think that module_folder was meant to be the name of a script to run.

answered Dec 23 '13 at 13:07

John Zwinck

239,568
38
324
436

3

That's what I want to do, but it says `getopt.GetoptError: option -m not recognized`. – Yajo Dec 26 '13 at 09:14
This is a very new feature and only works for python ~ 3.7 or later: https://github.com/python/cpython/pull/4752 – Evandro Coan May 19 '19 at 11:27
1

@user: I don't think so, as I wrote this answer in 2013, when Python 3.7 did not exist. Or, maybe this answer saw into the future by 5 years? – John Zwinck May 19 '19 at 13:56
2

It is simple, they saw your answer and copied it. When I tried you answer with CPython 3.6, it threw this error `getopt.GetoptError: option -m not recognized`. But when I updagraded to Python 3.7.2, it worked fine. – Evandro Coan May 19 '19 at 14:13

How to debug a python module that needs to be executed with -m?

2 Answers2