-1

I have a question about my dataset. I want to check which date + time is missing in my dataframe (see image). DF I want to know this for the columns named 'starttime' and 'endtime'.

How can I solve this?

I tried

pd.date_range(start = '2019-01-01 00:00:00', end = '2022-12-31 23:00:00' ).difference(allmerged.index)

but this is not working.

Trying the suggested code in the comments results in this output: output image

Timus
  • 10,974
  • 5
  • 14
  • 28
J1999
  • 3
  • 2
  • Please [do not post images of code, data, error messages, etc.](https://stackoverflow.com/help/how-to-ask) Add a [MRE](https://stackoverflow.com/help/minimal-reproducible-example) (also look [here](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples)) instead. – Timus Apr 13 '23 at 12:04

1 Answers1

0

First there is frequency 15Min in sample data, if need test this frequancy add parameter freq to date_range:

r = pd.date_range(start='2019-01-01 00:00:00', end='2022-12-31 23:00:00', freq='15Min')

If necessary convert both columns to datetimes:

df[['starttime','endtime']] = df[['starttime','endtime']].apply(pd.to_datetime)

Last testing by Index.difference:

#testing both together
out = r.difference(np.ravel(df[['starttime','endtime']].to_numpy()))

out = r.difference(df[['starttime','endtime']].stack())

#testing separately
start = r.difference(df['starttime'])
end = r.difference(df['endtime'])
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • Thanks! My data is indeed al converted to datetime. However, when I do the separate testing, I get two strange values. Namely: ['2022-12-31T23:00:00.000000000'] ['2019-01-01T00:00:00.000000000']. I will add a picture of this value to my question. – J1999 Apr 13 '23 at 08:39
  • @J1999 - I change solution with `Index.difference` – jezrael Apr 13 '23 at 08:40
  • With Index.difference, this is my output: DatetimeIndex(['2022-12-31 23:00:00'], dtype='datetime64[ns]', freq=None) DatetimeIndex(['2019-01-01'], dtype='datetime64[ns]', freq=None). Does that mean that there are no missing values? – J1999 Apr 13 '23 at 08:42
  • @J1999 - I think you are correct. – jezrael Apr 13 '23 at 08:46