Python Pandas date time drop row if older

Question

 LastUpdate            TS_UPDATE
0  2020-02-02  2019-10-30 15:27:20
6  2020-02-02  2019-10-30 15:27:14
8  2020-02-02  2019-10-30 15:27:07
9  2020-02-02  2019-10-30 15:27:07
10 2020-02-02  2019-10-30 15:27:07
11 2020-02-02  2019-10-30 15:27:05
12 2020-02-02  2019-10-30 15:27:04
13 2020-02-02  2019-10-30 15:27:03
14 2020-02-02  2019-10-30 15:27:03
15 2020-02-02  2019-10-30 15:27:02

How can I check if the LastUpdate is newer than TS_UPDATE? OR probably better: if TS_UPDATE is OLDER than LastUpdate if that isnt the case -> drop row.

Can anyone help me out?

Is it possible to do by boolean (> / <)?

 for row in df:
    if df["LastUpdate"][row] < df["TS_UPDATE"][row]:
        #drop row

Is the data already in datetime format? – hiranyajaya Mar 04 '20 at 15:54 — hiranyajaya, Mar 04 '20 at 15:54
This seems rather basic, have you read the Pandas docs? – AMC Mar 04 '20 at 16:00 — AMC, Mar 04 '20 at 16:00
yes it is already in datetime format – yannickhau Mar 04 '20 at 16:01 — yannickhau, Mar 04 '20 at 16:01

score 1 · Answer 1 · answered Mar 04 '20 at 15:57

1

The idiomatic way is to just keep the relevant rows:

resul = df[df["LastUpdate"] >= df["TS_UPDATE"]]

answered Mar 04 '20 at 15:57

Serge Ballesta

143,923
11
122
252

ipj · Answer 2 · 2020-03-04T16:16:09.603

0

You may use query method:

df.query("LastUpdate>=TS_UPDATE")

Very compact, SQL-like syntax.

If You insist on iteration:

for i, r in df.iterrows():
    if r.LastUpdate < r.TS_UPDATE:
        df.drop(i, inplace = True)

edited Mar 04 '20 at 16:16

answered Mar 04 '20 at 16:01

ipj

3,488
1
14
18

score 0 · Accepted Answer · answered Mar 04 '20 at 16:03

If the data is string, you would get an answer for the > or < comparison and here's how it works. If all the data in your dataframe is formatted in the way you have shown, the comparison should work. Instead of the for loop you can just do

new_df = df[df["LastUpdate"] < df["TS_UPDATE"]]

However, if you really want to compare the numbers, you can use pandas' pd.to_datetime function. This will transform the strings to datetime format which supports many numerical operations

df['LUdt'] = pd.to_datetime(df['LastUpdate'])
df['TSdt'] = pd.to_datetime(df['TS_UPDATE'])
new_df = df[df['LUdt'] > df['TUdt']]

Python Pandas date time drop row if older

3 Answers3