Detect consecutive dates in pandas series of DatetimeIndex

Question

I have pandas Series of DatetimeIndex in date format (YYYY-MM-DD) and want to label consecutive regions, where each index is consecutive in respect to a day - so if there is a missing date in a Datetime series, I want to detect it, i.e.:

...
2005-01-15
2005-01-16
2005-01-17
2005-02-15
2005-02-16
...

where a gap of missing days between 2005-01-17 and 2005-02-15 is evident.

Couldn't find easy way to do this with pandas, while I expect some helper function that I'm not aware of. More generally, also numpy solution would be appreciated.

@smci, I don't know what dput() is, but here is one way to generate sample data:

import pandas as pd
import numpy as np

data = pd.concat([
    pd.Series(np.random.randn(3), pd.date_range('2005-01-15', '2005-01-17')),
    pd.Series(np.random.randn(3), pd.date_range('2005-02-15', '2005-02-17'))
])

Thanks for adding the example. Doh! `dput()` is from R, not pandas, my brain thunked the wrong direction. — smci, Dec 28 '14 at 19:47
Near-dupe of: [Calculating time difference between two rows](http://stackoverflow.com/questions/25328125/calculating-time-difference-between-two-rows/), [Describing gaps in a time series pandas](http://stackoverflow.com/questions/24815720/describing-gaps-in-a-time-series-pandas) and [pandas TimeSeries diff() reverts to Series](http://stackoverflow.com/questions/24597446/pandas-timeseries-diff-reverts-to-series) — smci, Dec 28 '14 at 19:56

score 1 · Accepted Answer · edited May 23 '17 at 10:28

1

Try something like:

data.index - data.index.shift(1, freq=pd.DateOffset(1))

per @chrisb's answer to Calculating time difference between two rows

edited May 23 '17 at 10:28

Community

1
1

answered Dec 28 '14 at 19:37

smci

32,567
20
113
146

This now seems to do a set different between index index and the shifted index, which isn't what we're after here. How do you spell this in the newer Pandas? – Chris Withers Jul 27 '16 at 17:48
@ChrisWithers: by "now" do you mean "in Python 3.x" or "Pandas 0.17.x"? If we can narrow down when the change happened it would help... – smci Jul 28 '16 at 21:12
Python 2.x, Pandas 0.18.x – Chris Withers Jul 29 '16 at 05:59

score 0 · Answer 2 · answered Mar 19 '19 at 20:40

Smci's answer did not work for detecting missing date as the question was asking.

I use DataFrame.asfreq('D') to detect missing values. Those missing dates will be listed but their corresponding values will show NAN. For example:

df1 = df.asfreq('D)
missing_dates=df1[df1.Column.isnull()]

Detect consecutive dates in pandas series of DatetimeIndex

2 Answers2

Linked