Why is my date axis formatting broken when plotting with Pandas' built-in plot calls as opposed to via Matplotlib?

Question

I am plotting aggregated data in Python, using Pandas and Matlplotlib. My axis customization commands are failing as a function of which of two similar functions I'm calling to make bar plots. The working case is e.g.:

import datetime
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates

def format_x_date_month_day(ax):   
    days = mdates.DayLocator()
    months = mdates.MonthLocator()  # every month
    dayFmt = mdates.DateFormatter('%D')
    monthFmt = mdates.DateFormatter('%Y-%m')
    ax.figure.autofmt_xdate()
    ax.xaxis.set_major_locator(months) 
    ax.xaxis.set_major_formatter(monthFmt)
    ax.xaxis.set_minor_locator(days)

span_days = 90
start = pd.to_datetime("1-1-2012")
idx = pd.date_range(start, periods=span_days).tolist()
df=pd.DataFrame(index=idx, data={'A':np.random.random(span_days), 'B':np.random.random(span_days)})

plt.close('all')
fig, ax = plt.subplots(1)
ax.bar(df.index, df.A)      # loop over columns here to do stacked plot
format_x_date_month_day(ax)
plt.show()

(See matplotlib.org for example of looping to create a stacked bar plot.) This gives us

Another approach that should work and be much easier is to use df.plot.bar(ax=ax, stacked=True), however it does not admit date axis formatting with mdates:

plt.close('all')
fig, ax = plt.subplots(1)
df.plot.bar(ax=ax, stacked=True)
format_x_date_month_day(ax)
plt.show()

How can mdates and ax.figure.autofmt_xdate() be made to play nice with df.plot.bar?

Answer 1

Bar plots in pandas are designed to compare categories rather than to display time-series or other types of continuous variables, as stated in the docstring:

A bar plot shows comparisons among discrete categories. One axis of the plot shows the specific categories being compared, and the other axis represents a measured value.

This is why the scale of the x-axis of pandas bar plots is made of integers starting from zero, regardless of the data type of the x variable. When the same bar plot is created with matplotlib, the scale of the x-axis is made of matplotlib date numbers, so the tick locators and formatters of the matplotlib.dates module (mdates) can be used as expected.

To be able to use a pandas bar plot with mdates, you need to move the bars along the x-axis to locations that match the matplotlib date numbers. This can be done thanks to the mdates.date2num function. This is illustrated in the following example based on the code you provided with a few modifications: the sample dataset contains 3 variables, the time series is limited to 45 days, and the tick formatting is adjusted to my preferences (and is not wrapped as a function).

This example works for any number of variables (with or without NaNs) and for any bar width that is passed to the pandas plot function:

import numpy as np                   # v 1.19.2
import pandas as pd                  # v 1.1.3
import matplotlib.dates as mdates    # v 3.3.2

# Create random dataset
rng = np.random.default_rng(seed=1) # random number generator
nperiods = 45
nvar = 3
idx = pd.date_range('2012-01-01', periods=nperiods, freq='D')
df = pd.DataFrame(rng.integers(11, size=(idx.size, nvar)),
                  index=idx, columns=list('ABC'))

# Draw pandas stacked bar chart
ax = df.plot(kind='bar', stacked=True, figsize=(10,5))

# Compute width of bars in matplotlib date units
pandas_width = ax.patches[0].get_width() # the default bar width is 0.5
mdates_x0 = mdates.date2num(df.index[0])
mdates_x1 = mdates.date2num(df.index[1])
mdates_width_default = (mdates_x1-mdates_x0)/2
mdates_width = pandas_width*mdates_width_default/0.5 # rule of three conversion

# Compute new x values for bars in matplotlib date units, adjusting the
# positions according to the bar width
mdates_x = mdates.date2num(df.index) - mdates_width/2
nvar = len(ax.get_legend_handles_labels()[1])
mdates_x_patches = np.ravel(nvar*[mdates_x])

# Set bars to new x positions: this loop works fine with NaN values as
# well because in bar plot NaNs are drawn with a rectangle of 0 height
# located at the foot of the bar, you can verify this with patch.get_bbox()
for patch, new_x in zip(ax.patches, mdates_x_patches):
    patch.set_x(new_x)
    patch.set_width(mdates_width)

# Set major and minor date tick locators
months = mdates.MonthLocator()
days = mdates.DayLocator(bymonthday=np.arange(31, step=3))
ax.xaxis.set_major_locator(months)
ax.xaxis.set_minor_locator(days)

# Set major date tick formatter
month_fmt = mdates.DateFormatter('\n%b\n%Y')
day_fmt = mdates.DateFormatter('%d')
ax.xaxis.set_major_formatter(month_fmt)
ax.xaxis.set_minor_formatter(day_fmt)

# Shift the plot frame to where the bars are now located
xmin = min(mdates_x) - mdates_width
xmax = max(mdates_x) + 2*mdates_width
ax.set_xlim(xmin, xmax)

# Adjust tick label format last, else it may produce unexpected results
ax.figure.autofmt_xdate(rotation=0, ha='center')

Up to you to decide if this is more convenient than plotting stacked bars from scratch with matplotlib.

This solution can be slightly modified to display appropriate tick labels for time series based on any frequency of time. Here is an example using a frequency of minutes, a custom bar width, and an automatic date tick locator and formatter. Only the new/modified code lines are shown:

import matplotlib.ticker as mtick
#...

idx = pd.date_range('2012-01-01 12', periods=nperiods, freq='T')
#...

ax = df.plot(kind='bar', stacked=True, figsize=(10,5), width=0.3)
#...

# Set adaptive tick locators and major tick formatter
maj_loc = mdates.AutoDateLocator()
ax.xaxis.set_major_locator(maj_loc)
min_loc = mtick.FixedLocator(mdates_x + mdates_width/2)
ax.xaxis.set_minor_locator(min_loc) # draw minor tick under each bar
fmt = mdates.ConciseDateFormatter(maj_loc)
ax.xaxis.set_major_formatter(fmt)
#...

You may notice that the ticks are often not well aligned with the bars. There appears to be some issue with matplotlib when the figure elements are put together. I find this is usually only noticeable when plotting thinner-than-useful bars. You can check that the bars and ticks are indeed placed correctly by running ax.get_xticks() and comparing that to the values given by patch.get_bbox() when looping through ax.patches.