Why can't I select whole days from intra-day time series?

Question

I have basic financial OHLCV data.

                      Datetime        Open    Volume        date
0    2021-08-25 09:30:00-04:00  149.699997   3524920  2021-08-25
1    2021-08-25 09:35:00-04:00  149.699997   1424746  2021-08-25
2    2021-08-25 09:40:00-04:00  149.785004   1248013  2021-08-25
3    2021-08-25 09:45:00-04:00  149.649994   1208242  2021-08-25
4    2021-08-25 09:50:00-04:00  149.486603   1345607  2021-08-25

Let's say I want to select all the examples from one specific day.

df["Datetime"].eq("2021-08-25")

This doesn't work even though comparisons to string date work when you use lt or gt.

So I created the date column with

df["date"] = df["Datetime"].dt.date

This new column is than of object dtype yet:

df_n["date"].eq("2021-08-25")

Still doesn't work.

jezrael · Accepted Answer · 2021-11-18T11:18:57.133

2

One solution is 'remove' times, it means set them to 00:00:00 by Series.str.normalize, because obviously pandas compare strings datetimes with timestamps:

df["Datetime"].dt.normalize().eq("2021-08-25")
df["Datetime"].dt.floor('d').eq("2021-08-25")

If need compare dates, need dates in another side too:

from datetime import date

df["Datetime"].dt.date.eq(date(2021,8,25))

edited Nov 18 '21 at 11:18

answered Nov 18 '21 at 11:16

jezrael

822,522
95
1,334
1,252

Thanks. I still find it strange as .dt.date returns a column with object dtype a string basically. I don't know why I can't compare. – Borut Flis Nov 18 '21 at 11:18
@BorutFlis - yes, because `date`s are python objects. – jezrael Nov 18 '21 at 11:19

score 2 · Answer 2 · answered Nov 18 '21 at 11:21

2

Using strftime can also do

df1['Datetime']= pd.to_datetime(df1['Datetime'] )
df1[df1['Datetime'].dt.strftime('%Y-%m-%d')=='2021-08-25']

answered Nov 18 '21 at 11:21

wwnde

26,119
6
18
32

Why can't I select whole days from intra-day time series?

2 Answers2