Introduction and examples.
It is a widely known best practice to close matplotlib figures after opening them to avoid consuming too much memory. In a standalone Python script, for example, I might do something like this:
fig, ax = plt.subplots()
ax.plot(x, y1);
plt.show() # stop here and wait for user to finish
plt.close(fig)
Similarly, for a Jupyter notebook it's a common pattern to create a figure in one cell and then close it in the next cell. A minimal example might look something like this, where each blank line is a new cell:
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 100)
y1 = np.sin(x)
fig, ax = plt.subplots()
ax.plot(x, y1);
plt.close(fig)
However, this has a distinct disadvantage:
the figure is blank when run with "Kernel" -> "Restart & Run All"
or "Cell" -> "Run All".
That is, the figure doesn't display until all cells finish evaluating,
but the figure doesn't have time to render
since we are using plt.close right afterward.
Here's an example screenshot based on the example above:
This is different from running each cell individually: as long as I run each cell slowly enough, I can get each figure to display:
Full Jupyter notebook .ipynb files are available here:
https://github.com/nbeaver/jupyter-figure-rendering-tests
Inadequate workarounds.
The simplest workaround is start from the top and manually step through each cell, wait for it to render, and then move onto the next cell. I don't consider this an acceptable workaround, as it becomes impractically laborious for large notebooks and is generally contrary to the purpose of an executable notebook environment.
Another workaround is to simply never call plt.close(fig).
This is not a good option for several reasons:
It results in excess memory usage, as evidenced by warnings about opening too many figures. On some machines or resource-constrained environments this may prevent the notebook execution from completing at all.
Even in environments with abundant memory, cursor tracking and interactive functionality like zoom or pan becomes very slow when many figures are open at once.
As mentioned above, in general none of the figures will display until the final cell has finished executing, which is undesirable for notebooks where execution time may be long for certain cells further down.
Another workaround is to use %matplotlib inline
instead of %matplotlib notebook.
This is unsatisfactory also:
The
inlinemode does not permit interactive inspection of the figure such as cursor position values or pan and zoom. This functionality may be desirable or essential for analysis.The
inlineandnotebooksettings cannot in general be toggled on a per-cell basis, so effectively this is an all-or-nothing setting.Even if it were possible to toggle between
inlineandnotebook, I would prefer to only usenotebookand then close the figure in a subsequent cell, so that I can return to the cell later and re-run it to get the interactive controls without needing to edit the cell.
An analogous workaround is to call savefig for each cell with a figure
and then browse the generated images with an external image viewing program.
While this allows limited zooming and panning,
it doesn't give cursor positions
and it's really not comparable to the interactive notebook plots.
Criteria and current workaround.
Here are my requirements:
The effect of "Restart & Run All" must render all figures eventually; no figures can be left blank.
Use
%matplotlib notebookfor all cells so that I can re-run the cell later and inspect the figure interactively.Allow
plt.close(fig)after each cell so that notebook resources can be conserved.
Essentially, I would like a way to force the kernel
to render the current figure
before proceeding on to plt.close(fig).
My hope is that this is a well-known behavior
and that I've simply missed something.
Here's what I have tried so far that didn't help at all:
plt.show()at the end of a cell or between cells.fig.show()at the end of a cell or between cells.plt.ioff()at the end of a cell or between cells.time.sleep(1)at the end of a cell or between cells.plt.pause(1)at the end of a cell or between cells.fig.canvas.draw_idle()at the end of a cell or between cells.Doing
from IPython.display import displayand thendisplay(fig)at the end of a cell or between cells.calling
plt.close('all')at end of notebook instead of between each cell.
Currently, the best I've been able to do
is call fig.canvas.draw() in a separate cell
between the figure and the cell with plt.close(fig).
Using the example above, here's what this looks like:
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 100)
y1 = np.sin(x)
fig, ax = plt.subplots()
ax.plot(x, y1);
fig.canvas.draw()
plt.close(fig)
This works reliably in smaller notebooks,
and for a while I thought this had solved the problem.
However, this doesn't always work;
particularly in large notebooks with many figures,
some of them still come out blank,
suggesting that fig.canvas.draw() adds some delay
but is not always sufficient,
perhaps due to a race condition.
Since fig.canvas.draw() is not a documented method
for making Jupyter notebooks render the current figure,
I would hesitate to describe this as a bug,
although it seems to be the closely related to this matplotlib issue,
which ultimately seems to be a Jupyter bug:
The simplest work around may be to put the
input()call in the next cell or to addplt.gcf().canvas.draw()above the input call. This will still result in "dead" figures (which may not have caught up to the hi-dpi settings of your browser), but they will at least show.
I've observed this behavior in many combinations of matplotlib and Jupyter, including matplotlib version 2.1.1 and 3.5.1 and Jupyter version 4.4.0 and 6.4.8. I've also observed it in both Google Chrome 99.0.4844.51 and Firefox 97.0.2 and on both Windows 10 and Ubuntu 18.04.6.
Related questions (not duplicates):
