Avoid restarting Jupyter kernel in package develop mode?

Question

I am working on a python package as a developer. The package is stored in a git repo and I use a local branch to debug/fix this package.

I use Jupyter notebooks using pip in edit mode to load my local branch as a package, where I test changes I make.

I run the following steps:

1. Load the local package in a develop mode
2. Import the module I want to test
3. Do the test

For instance:

! pip install -e Path/To/Local/Package/ # step 1

import local_foo as foo # step 2
foo.print() # step 3

After step 3 if the code doesn't behave as expected, I correct my package, restart the jupyter kernel, and re-run the 3 previous steps until I get the behavior I want.

My question is:
Is there a way to avoid restarting the kernel?

I tried the following but it doesn't work in this case:
IPython autoreload:

%load_ext autoreload
%autoreload 2

and importlib.reload:

import importlib
importlib.reload(foo)

I tried the solution suggested in this article:
https://support.enthought.com/hc/en-us/articles/204469240-Jupyter-IPython-After-editing-a-module-changes-are-not-effective-without-kernel-restart

Many thanks!

---

PS: In addition, can some of you can share tips, workflows or experiences using Jupyter notebooks to manage python package development (test,...)

Answer 1

I faced a similar issue , while importing a custom script in jupyter notebook

Try importing the module as an alias then reloading it

import local_foo as foo
from importlib import reload
reload(foo)

Answer 2

I hate restarting kernels and sometimes it seems like it is the only option but I have tried this and it worked:

import local_foo as foo
from importlib import reload

reload(foo)

Answer 3

You may have to deal with reference issues as packages which are not pure Python may not be easily reloaded using importlib.reload.

This may be resolved by removing it from the modules and namespace before reloading it.

import foo  # The first import of the package

import sys
import importlib

# Do things with foo

del sys.modules['foo']  # Remove it from the currently loaded modules
del foo  # Remove it's name

importlib.reload(foo)

Joseph Garvin writes a very nice function in this answer which operates similarly and also supports imports of the form from foo import bar.

Answer 4

You can use autoreload, adding these two lines at the beginning of your Jupyter notebook:

%reload_ext autoreload
%autoreload 2