How to filter df by a certain date?

Question

I have a df of data. I need to be able to return certain rows of the data based on the day of the week.

If its monday I need to print the rows which contain data for the past 3 days. If its wednesday or friday I need to print the rows which contain the data for the past two days.

Merging two df to create one:

df_new = pd.concat([outcomes_df, specialists_df], ignore_index=True)

df_new['Published Date'] = pd.to_datetime(df_new['Published Date'])

Getting the correct dates based today's date:

N=0
if datetime.today().weekday() == 0:
    N = 3
elif datetime.today().weekday() == 2 or datetime.today().weekday() == 4:
    N = 2
else:
    pass

mydate = datetime.now() - timedelta(days=N)
print(mydate)

Filtering by the date range

df_new = df_new[(df_new['Published Date'] >= mydate) & (df_new['Published Date'] <= datetime.today())]

print(mydate) results in the correct date

print(df_new) results in error:

Empty DataFrame Columns: [col1, col2, col3, Published Date] Index: []

Welcome to StackOverflow. To be able to help you better, you should provide the data you are working with and what the expected output looks liks. Find more information here: https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples — Erfan, Apr 02 '19 at 15:54

score 0 · Answer 1 · answered Apr 02 '19 at 15:55

You do not need a if else here , since that will need for loop as well, here I am using np.select

import numpy as np
s=df_new['Published Date'].dt.weekday
d=np.select([s==0,s.isin([2,4])],[3,2],0)


df_new['mydate']=(pd.to_datetime('today')-pd.to_timedelta(d,unit='D'))

df_new = df_new[(df_new['Published Date'] >= df_new['mydate']) & (df_new['Published Date'] <= datetime.today())]

How to filter df by a certain date?

1 Answers1