Animation doesn't work with datetime X values, but it works properly with integers

Question

I'm trying to create an animated plot using matplotlib. It works as expected when i'm using integers for the X values:

#!/usr/bin/env python
import os
import random
import numpy as np
from datetime import datetime as dt, timedelta
from collections import deque

import matplotlib.pyplot as plt  # $ pip install matplotlib
import matplotlib.animation as animation

%matplotlib notebook

npoints = 30
x = deque([0], maxlen=npoints)
y = deque([0], maxlen=npoints)
fig, ax = plt.subplots()
[line] = ax.plot(x, y)

def get_data():
    t = random.randint(-100, 100)
    return t * np.sin(t**2)

def data_gen():
    while True:
        yield get_data()

def update(dy):
    x.append(x[-1] + 1)
    y.append(dy)
    line.set_data(x, y)
    ax.relim()
    ax.autoscale_view(True, True, True)
    return line, ax

plt.rcParams['animation.convert_path'] = 'c:/bin/convert.exe'
ani = animation.FuncAnimation(fig, update, data_gen, interval=500, blit=True)
#ani.save(os.path.join('C:/','temp','test.gif'), writer='imagemagick', fps=30)
plt.show()

this produces the following animation:

however as soon as i'm trying to use datetime values as x values - the plot is empty:

npoints = 30
x = deque([dt.now()], maxlen=npoints)   # NOTE: `dt.now()`
y = deque([0], maxlen=npoints)
fig, ax = plt.subplots()
[line] = ax.plot(x, y)

def get_data():
    t = random.randint(-100, 100)
    return t * np.sin(t**2)

def data_gen():
    while True:
        yield get_data()

def update(dy):
    x.append(dt.now())                  # NOTE: `dt.now()`
    y.append(dy)
    line.set_data(x, y)
    ax.relim()
    ax.autoscale_view(True, True, True)
    return line, ax

plt.rcParams['animation.convert_path'] = 'c:/bin/convert.exe'
ani = animation.FuncAnimation(fig, update, data_gen, interval=1000, blit=True)
#ani.save(os.path.join('C:/','temp','test.gif'), writer='imagemagick', fps=30)
plt.show()

what am I doing wrong?

PS I'm using matplotlib version: 2.1.2

Have you checked this answer: https://stackoverflow.com/questions/1574088/plotting-time-in-python-with-matplotlib? It involves converting the datetime objects into numbers using date2num before plotting the chart. — M. F., Mar 20 '18 at 18:26
@ImportanceOfBeingErnest, thank you for the link! I've tried `date2num ` - the plot is still empty. Did it work for you with datetimes? Which matplotlib version do you use? — MaxU - stand with Ukraine, Mar 20 '18 at 19:29
It wasn't me who gave you that link, because I would not expect a date2num conversion be necessary here. I initially tested this with 2.2.0, but now I also tested with 2.1.0. Both codes work fine in both versions. — ImportanceOfBeingErnest, Mar 20 '18 at 19:39

ImportanceOfBeingErnest · Accepted Answer · 2018-03-20T20:33:50.410

The code from the question runs fine for me in matplotlib 2.2.0 in a Jupyter notebook (%matplotlib notebook). It does fail however using any of the following backends when run as script: Qt4Agg, Qt4Cairo, TkAgg, TkCairo.

I would hence suspect that @M.F.'s comment above is indeed true and that date2num conversion is necessary.

This is what the following code does, apart from getting rid of the blitting, which is not useful in the case where the axes itself has to be drawn as well.

import random
import numpy as np
from datetime import datetime as dt, timedelta
from collections import deque
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.animation as animation

npoints = 30
x = deque([mdates.date2num(dt.now())], maxlen=npoints)   # NOTE: `dt.now()`
y = deque([0], maxlen=npoints)
fig, ax = plt.subplots()
[line] = ax.plot_date(x, y, ls="-", marker="")

def get_data():
    t = random.randint(-100, 100)
    return t * np.sin(t**2)

def data_gen():
    while True:
        yield get_data()


def update(dy):
    x.append(mdates.date2num(dt.now()))                  # NOTE: `dt.now()`
    y.append(dy)
    line.set_data(x, y)
    ax.relim()
    ax.autoscale_view(True, True, True)    

ani = animation.FuncAnimation(fig, update, data_gen, interval=1000)
#ani.save("anidates.gif", writer='imagemagick', fps=30)
plt.show()

That may be the point, I tested initially with the notebook backend in jupyter where the code worked fine, but I then tried the Qt4Agg backend in a script and there it failed. Thje code from this answer-attempt however works in all cases. Does it work for you? — ImportanceOfBeingErnest, Mar 20 '18 at 20:18
@ImportanceOfBeingErnest, it works, thank you! I'm trying to understand why it didn't work for me before, when I tried to use `date2num`... — MaxU - stand with Ukraine, Mar 20 '18 at 20:21
Your code works fine with `GTK3Cairo` and `Qt5Agg`. MaxU's original code worked fine for me (using matplotlib v2.2.0) with `GTK3Cairo` but with backend `Qt5Agg` the xlabels do not update. — unutbu, Mar 20 '18 at 20:28

unutbu · Answer 2 · 2018-03-22T20:29:07.877

Using pandas, you could register a converter (by calling register_matplotlib_converters()) to tell matplotlib how to handle datetime.datetime objects when line.set_data is called so that you do not have to call date2num on each value yourself:

import datetime as DT
import collections 
import random
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.dates as mdates
import pandas.plotting as pdplt
pdplt.register_matplotlib_converters()

npoints = 30
x = collections.deque([DT.datetime.now()], maxlen=npoints)   
y = collections.deque([0], maxlen=npoints)
fig, ax = plt.subplots()
[line] = ax.plot(x, y)
# Not necessary, but offers more control over the format
xfmt = mdates.DateFormatter('%H:%M:%S')
ax.xaxis.set_major_formatter(xfmt)

def get_data():
    t = random.randint(-100, 100)
    return t * np.sin(t**2)

def data_gen():
    while True:
        yield get_data()

def update(dy):
    x.append(DT.datetime.now())                  
    y.append(dy)
    line.set_data(list(x), y)
    ax.relim()
    ax.autoscale_view()
    # Not necessary, but it rotates the labels, making them more readable
    fig.autofmt_xdate()
    return [line]

ani = animation.FuncAnimation(fig, update, data_gen, interval=1000, blit=False)
plt.show()

Tested with matplotlib version 2.2.0, backends TkAgg, Qt4Agg, Qt5Agg, GTK3Agg, and GTK3Cairo.

matplotlib.units maintains a registry of converters which it uses to convert non "numlike" values to plottable values.

In [91]: import matplotlib.units as munits
In [92]: munits.registry
Out[92]: 
{numpy.str_: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc99e8>,
 numpy.bytes_: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc9a58>,
 str: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc9a20>,
 bytes: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc99b0>}

High-level plot functions like plt.plot handle datetimes automatically, but lower-level methods like line.set_data do not. So if we want to make an animation which uses datetime objects, and we do not wish to call date2num manually on each value, then we could instead register a converter.

If we have pandas installed, then instead of writing a converter from scratch, we could use pandas.plotting.register_matplotlib_converters, which teaches matplotlib to handle (among other things) lists of datetime.datetime objects.

In [96]: import pandas.plotting as pdplt
In [97]: pdplt.register_matplotlib_converters()
In [98]: munits.registry
Out[98]: 
{datetime.datetime: <pandas.plotting._converter.DatetimeConverter at 0x7f1d400145f8>,
 numpy.str_: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc99e8>,
 numpy.bytes_: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc9a58>,
 pandas._libs.tslibs.timestamps.Timestamp: <pandas.plotting._converter.DatetimeConverter at 0x7f1d40014668>,
 str: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc9a20>,
 numpy.datetime64: <pandas.plotting._converter.DatetimeConverter at 0x7f1d400142b0>,
 datetime.date: <pandas.plotting._converter.DatetimeConverter at 0x7f1d40014748>,
 datetime.time: <pandas.plotting._converter.TimeConverter at 0x7f1d40014240>,
 bytes: <matplotlib.category.StrCategoryConverter at 0x7f1d65cc99b0>,
 pandas._libs.tslibs.period.Period: <pandas.plotting._converter.PeriodConverter at 0x7f1d40014710>}

Unfortunately, DatetimeConverter it does not handle deques of datetime.datetime objects.

To get around this little roadblock, call

line.set_data(list(x), y)

instead of

line.set_data(x, y)

Thank you! It works properly in Jupyter. In `iPython` (`backend: Qt5Agg`, matplotlib 2.1.2) it gives me `TypeError: float() argument must be a string or a number, not 'datetime.datetime'` — MaxU - stand with Ukraine, Mar 20 '18 at 20:50
That's interesting. I'll try compiling 2.1.2 to see if I can reproduce the situation. At the moment all I can say is that it seems to work for me using 2.2.0 and Qt5Agg. — unutbu, Mar 20 '18 at 21:00
My mistake. I can reproduce the problem in IPython, with 2.1.2 (and also 2.2.0). Sorry I don't know how to fix it. — unutbu, Mar 20 '18 at 21:39
I think registering a converter might be an alternative to calling `date2num` manually. I've edited the code above to show what I mean. I tested it successfully as a script, and as %cpasted code in IPython with backends TkAgg, Qt4Agg, Qt5Agg, GTK3Agg, and GTK3Cairo. *I did not work for me in jupyter notebook*, but I'm hoping it might work for you since I suspect my jupyter is not compiled or configured correctly... — unutbu, Mar 22 '18 at 20:01
thanks a lot! I checked it - it works perfectly in the `ipython` (backend: `Qt5Agg`). I'll add it to my collection of recipes... PS unfortunately I can't upvote your solution one more time ;-) — MaxU - stand with Ukraine, Mar 22 '18 at 20:17

Animation doesn't work with datetime X values, but it works properly with integers

2 Answers2