-1

I need to filter a stock file for the first 3 years, 2016-2018. My data goes to 2020. I keep getting the same df for all years. This is what I have tried:

df3 = (df2['Date'] > '01-01-2016') & (df2['Date'] <= '31-12-2018')

df3 = (df2['Date'] > '01-01-2016') & (df2['Date'] <= '31-12-2018')

df3 = df2.loc[df3]

df3

It has to be in how I am filtering or the operators, right? I have tried so many variations and either get the same data, or an empty df.

Anurag Dabas
  • 23,866
  • 9
  • 21
  • 41
Neonleon
  • 25
  • 4
  • convert 'Date' to datetime then use `df3 = (df2['Date'] > pd.to_datetime('01-01-2016')) & (df2['Date'] <= pd.to_datetime('31-12-2018'))` – Anurag Dabas Jul 15 '21 at 03:20
  • 2
    pandas Series have a [`.between()`](https://pandas.pydata.org/docs/reference/api/pandas.Series.between.html) method. So do `df2['Date'].between('01-01-2016', '31-12-2018')`. Make sure your column is a datetime. @AnuragDabas and everyone. – smci Jul 15 '21 at 03:28
  • All the options mentioned in the above comments as well as how to convert to datetime is listed in the answers in the linked duplicate. – Henry Ecker Jul 15 '21 at 03:31

1 Answers1

-1

Extract the year from a datetime field first:

df['Year'] = [i.year for i in df['Date']]

And then you could try something like this:

df1 = df.loc[df['Year'] <= 2018]

Let me know if this was helpful.

TeflonMusk
  • 130
  • 8
  • If you're going to do this approach it should use the datetime accessor [`dt.year`](https://pandas.pydata.org/docs/reference/api/pandas.Series.dt.year.html) -> `df.loc[df['Date'].dt.year <= 2018]` not a comprehension. But this does not assess all the conditions given in the OP. – Henry Ecker Jul 15 '21 at 03:25
  • @HenryEcker This would be one approach as well. Just gave a snippet that could be used to get OP going. Not difficult to adjust to fill the remaining conditions. – TeflonMusk Jul 15 '21 at 03:28
  • pandas Series have a [`.between()`](https://pandas.pydata.org/docs/reference/api/pandas.Series.between.html) method, for datetimes. No need for piecewise comparing or iterating. – smci Jul 15 '21 at 03:29