how to avoid reloading package when rerun debugger

Question

Because I imported many packages, every time when I run my script in debug mode, it takes around a minute for me to hit my first line of code.

I run in debug mode many times a day, so I spend quite a bit time waiting.

Is there any way to speed it up?

How are you importing packages? for example are you doing "import json" or "from json import loads" Obviously the latter is much quicker as it only imports what you need. — C. Fennell, Oct 26 '18 at 23:17
I could envision writing a small wrapper to load the imports and parse input from the terminal to a call to `exec`. This is a hacky solution and probably the wrong answer. you would run the wrapper like a normal python script and then use it to call the script you are testing. — Stephen, Oct 26 '18 at 23:24
I am doing from json import loads. The problem is I am importing from other folk's package which also imports someone else' package. They might doing improt *, but I have no control over this.. — Lisa, Oct 26 '18 at 23:24

unutbu · Answer 1 · 2018-10-27T02:40:22.443

One way to speed up the development cycle is to preserve your Python process so you only need to load the packages once.

Instead of running the script, package the script itself as a module. Open an interactive python (or perhaps IPython) session, import your module, then "run the script" by calling its main function.

If you make changes to the script, you will only have to reload your module. Since the other modules are already loaded, this should be relatively quick. Since Python modules are cached, importing the same module a second time is essentially instantaneous.

In Python, reloading can be done this way. Or you could configure IPython to automatically reload modules or packages when they change this way. IPython also has some nice debugging features, such as %pdb, which will drop you into a debugger whenever an uncaught exception is raised.

So instead of your script.py looking like this:

import xyz

statement1
statement2
statement3

You'd modify it to look like

import xyz

def main():
    statement1
    statement2
    statement3

# This allows you to still run your module as a script:
if __name__ == '__main__':
    main()

Then, at the interactive Python session prompt, you'd type

>>> import script   
>>> script.main()

to run the script.

how to avoid reloading package when rerun debugger

1 Answers1