Strange crash after saving many colormeshplots using matplotlib

Question

I am using matplotlib(with LaTex enabled and agg backend) in order to display hydrodynamical simulations using the pcolormesh function. System: Ubuntu 16.04 Python: Python 2.7.12 Matplotlib: 1.5.1-1ubuntu1 Im am also adding additional axis using twinx twiny After saving ~300 png images I get a crash pointing to a non-working dvi conversion command when processing the title. I am always using pylab.close("all") in order to close my figure after each saving process.

pylab.rc('text', usetex=True)
pylab.rc('text.latex', preamble=[r"\usepackage{amsmath}", r"\usepackage{amssymb}", r"\usepackage[alsoload=astronomy]{siunitx}"])

pylab.close("all")
f, ax = pylab.subplots(1)
cmap = pylab.cm.jet
cmap.set_bad('w',1.)
if in logquantity:
    plot = ax.pcolormesh(x, y, quantity , vmax=maxvalue, vmin=minvalue, norm=LogNorm(vmax= maxvalue, vmin = minvalue), cmap=cmap)
else:
    plot = ax.pcolormesh(x, y, quantity, vmax=maxvalue, vmin=minvalue, cmap=cmap)
ax.set_title("Title")
ax.set_aspect("equal")
ax.set_xscale("log")
ax.grid()
...<More Formatting stuff>
ax2 = ax.axes.twiny()
ax3 = ax.twinx()
...<Formatting labels, setting scales etc>
pylab.savefig("test.png", dpi = 800, bbox_inches='tight')
pylab.close("all")

pylab.savefig("test.png", dpi = 800, bbox_inches='tight')
  File "/usr/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 689, in savefig
    fig.canvas.draw_idle()   # need this if 'transparent=True' to reset colors
  File "/usr/lib/python2.7/dist-packages/matplotlib/backend_bases.py", line 2026, in draw_idle
    self.draw(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.py", line 474, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/figure.py", line 1159, in draw
    func(*args)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/axes/_base.py", line 2324, in draw
    a.draw(renderer)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/text.py", line 792, in draw
    mtext=mtext)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.py", line 253, in draw_tex
    Z = texmanager.get_grey(s, size, self.dpi)
  File "/usr/lib/python2.7/dist-packages/matplotlib/texmanager.py", line 583, in get_grey
    pngfile = self.make_png(tex, fontsize, dpi)
  File "/usr/lib/python2.7/dist-packages/matplotlib/texmanager.py", line 524, in make_png
    'dvipng: \n\n' % dvifile + report)
RuntimeError: dvipng was not able to process the following file:
$HOME/.cache/matplotlib/tex.cache/2664bc1e879cb69289e1663e63a1c713.dvi
Here is the full report generated by dvipng:

No dvipng error report available.

If I check the mentioned .cache location I can perfectly find the dvi file mentioned. if I use dvipng on it I get the perfectly correctly rendered header it should put in the plot. I am assuming some kind of cache/buffer is running full. If it only concerns lineplots using plot,semilogy, semilogx and loglog I get this error after a much higher number of images. I can also provided the formatting details(or any other additional information) if anoyone thinks they are relevant for this problem.

Thank you in advance for your answers :-)

EDIT: The string is $\rho$$\left[6.96e-20 \frac{\si{\gram}}{\si{\centi \meter \cubed}}\right]$, $t =\num{0.00e+00}\text{y}$, $n_R = \num{300}$, $n_{\varphi} = \num{50}$, $n_z = \num{401},\ [C.L.] = 1.14e+02\text{pc}$

Png Version of cached file This is the result when I copy the cached file to another location and convert it to a png manually using dvipng.

The problem is not specific to this plot. It can happen with any plot. It works fine for whatever the first ~300 plots are. It is not always the same plot for which it crashes. That is one of the problems of looking into the problem. There is a loop that runs through all quantities and then calls functions to plot

for quantityname in quantities:
    quantitydict = quantities[quantityname]
    quantity = quantitydict["data"]

There are 18 of these quantities per time-step. EDIT2(More Information) The error occurs more often when the machine it is run on does not have much free Main Memory(below 10GB, script uses roughly 1GB while running). Also, sometimes manifests differently. It gives a LaTeX error. All the previous plots created in the run work just fine and I copy the line and compile it in LaTeX I get the correct result. Error Message:

    pylab.savefig("test.png", dpi = 800, bbox_inches='tight')
  File "/usr/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 688, in savefig
    res = fig.savefig(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/figure.py", line 1565, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backend_bases.py", line 2180, in print_figure
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.py", line 527, in print_png
    FigureCanvasAgg.draw(self)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.py", line 474, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/figure.py", line 1159, in draw
    func(*args)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/axes/_base.py", line 2324, in draw
    a.draw(renderer)
  File "/usr/lib/python2.7/dist-packages/matplotlib/artist.py", line 61, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/matplotlib/text.py", line 749, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "/usr/lib/python2.7/dist-packages/matplotlib/text.py", line 361, in _get_layout
    ismath=ismath)
  File "/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.py", line 229, in get_text_width_height_descent
    renderer=self)
  File "/usr/lib/python2.7/dist-packages/matplotlib/texmanager.py", line 675, in get_text_width_height_descent
    dvifile = self.make_dvi(tex, fontsize)
  File "/usr/lib/python2.7/dist-packages/matplotlib/texmanager.py", line 422, in make_dvi
    report))
RuntimeError: LaTeX was not able to process the following string:
'CFL $\\\\Delta t_\\\\text{min}$$\\\\left[4.65e+05 \\\\text{yrs}\\\\right]$, $t =\\\\num{3.80e+03}\\\\text{y}$, $n_R = \\\\num{300}$, $n_{\\\\varphi} = \\\\num{50}$, $n_z = \\\\num{401},\\\\ [C.L.] = 1.14e+02\\\\text{pc}$'
Here is the full report generated by LaTeX: 

No latex error report available.

Is this error related to this specific plot? Will it work, if you process this plot alone or first in the loop? What does the string to render look like? — ImportanceOfBeingErnest, Nov 19 '16 at 16:40
We sometimes see transient failures like this in the CI mpl runs an travis. My suspicion is that this is a filesystem related race condition (the dvi file is not fully flushed to disk before we try to re-read it to convert to png, when you look later everything is hunky-dory). How are you running the loop? — tacaswell, Nov 20 '16 at 20:11
I added the request additional information. Please ask if I can provide more things to help. — F. Klein, Nov 21 '16 at 17:24
Updated the question with even more Information that might be relevant. — F. Klein, Dec 06 '16 at 17:13

Strange crash after saving many colormeshplots using matplotlib

0 Answers0