3

I'm trying to plot a pandas series, but I'm encountering an error when I attempt to format the x-axis date.

(A related issue was identified in the comments, but it appears that it was resolved in a much older version of pandas than what I'm using. So, it seems like this is a new problem.)

Consider a plot of the following pandas series:

import pandas as pd

d = {pd.Timestamp('2021-03-15 08:30:00'): -65.926651,
     pd.Timestamp('2021-03-15 08:30:05'): -42.115551,
     pd.Timestamp('2021-03-15 08:30:10'): -24.699627,
     pd.Timestamp('2021-03-15 08:30:15'): -12.010081,
     pd.Timestamp('2021-03-15 08:30:20'): -2.781321}

s = pd.Series(d)

ax = s.plot()

I seek to format the x-axis date on the plot using:

from matplotlib.dates import DateFormatter
format_str: str = '%H:%M:%S'
format_: DateFormatter = DateFormatter(format_str)
ax.xaxis.set_major_formatter(format_)

This results in the following error:

      Traceback (most recent call last):
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/backends/backend_macosx.py", line 61, in _draw
    self.figure.draw(renderer)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/figure.py", line 1863, in draw
    mimage._draw_list_compositing_images(
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/cbook/deprecation.py", line 411, in wrapper
    return func(*inner_args, **inner_kwargs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/axes/_base.py", line 2747, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/axis.py", line 1164, in draw
    ticks_to_draw = self._update_ticks()
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/axis.py", line 1022, in _update_ticks
    major_labels = self.major.formatter.format_ticks(major_locs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/ticker.py", line 250, in format_ticks
    return [self(value, i) for i, value in enumerate(values)]
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/ticker.py", line 250, in <listcomp>
    return [self(value, i) for i, value in enumerate(values)]
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/dates.py", line 605, in __call__
    return num2date(x, self.tz).strftime(self.fmt)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/dates.py", line 511, in num2date
    return _from_ordinalf_np_vectorized(x, tz).tolist()
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/numpy/lib/function_base.py", line 2108, in __call__
    return self._vectorize_call(func=func, args=vargs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/numpy/lib/function_base.py", line 2192, in _vectorize_call
    outputs = ufunc(*inputs)
  File "/Users/me/VirtualEnvironments/my_venv/lib/python3.9/site-packages/matplotlib/dates.py", line 331, in _from_ordinalf
    np.timedelta64(int(np.round(x * MUSECONDS_PER_DAY)), 'us'))
OverflowError: int too big to convert

Interestingly, if I add a fractional offset to the timestamps, everything works:

s.index += pd.DateOffset(seconds=0.5)

When I examine x in the np.timedelta64 call, it corresponds to the number of days since the start of the unix epoch (1 Jan 1970) only if I add a fractional part to the timestamp. If there's no fractional part, the resulting integer is huge and seems to have no obvious relationship to the number of days since 1 Jan 1970.

What's wrong here?

rhz
  • 960
  • 14
  • 29
  • Does this answer your question? [Why do I get "python int too large to convert to C long" errors when I use matplotlib's DateFormatter to format dates on the x axis?](https://stackoverflow.com/questions/26642133/why-do-i-get-python-int-too-large-to-convert-to-c-long-errors-when-i-use-matpl) – SiHa Mar 16 '21 at 19:16
  • hmm but according to that, the bug was fixed in 0.15.1 ([released Nov 2014](http://pandas-docs.github.io/pandas-docs-travis/whatsnew/v0.15.1.html)) – tdy Mar 16 '21 at 19:52
  • @SiHa, I am using a much newer version of pandas: 1.1.5. – rhz Mar 16 '21 at 21:34
  • It's not `pandas`. It's `matplotlib`, – Tim Roberts Mar 16 '21 at 22:23
  • the link says it's a `matplotlib` error caused by `pandas` 0.15.0 and fixed in `pandas` 0.15.1 – tdy Mar 16 '21 at 22:34
  • Added my versions of ```matplotlib``` and ```pandas``` to question. – rhz Mar 16 '21 at 23:10

2 Answers2

2

Error occurred because data was given that exceeded the number range handled by DateFormatter.

Please refer to the official reference.

For example, the actual data for the first time series looks like this

s.index[0].value
1615797000000000000

This needs to be converted to numbers that can be handled by matplotlib.

s.index = mdates.date2num(s.index)
s
18701.354167   -65.926651
18701.354225   -42.115551
18701.354282   -24.699627
18701.354340   -12.010081
18701.354398    -2.781321
dtype: float64

update(I am on 3.6.3, so I am fixing it.)

ax = s.plot(style='o-')
import matplotlib.dates as mdates
format_str = '%H:%M:%S'
format_ = mdates.DateFormatter(format_str)
ax.xaxis.set_major_formatter(format_)

enter image description here

r-beginners
  • 31,170
  • 3
  • 14
  • 32
  • I don't quite understand. matplotlib appears to handle s.index fine when there's a fractional second component. When there's a fractional second, the plot also has the timestamps on a 45 deg angle (which is want I wanted) – rhz Mar 18 '21 at 17:20
-1

I had the same error message and I used this to resolve it.

import matplotlib.dates as mdates
ts = mdates.epoch2num(ts)

After that, I didn't have any error. I hope it will be help you.

B2RJ
  • 1
  • 8
    To be clear, how do I use this solution if I have a pandas series I wish to plot? – rhz Jul 02 '21 at 14:05