Modify pandas dataframe cells by filtering datatime details

Question

I have a dataset with index, timestamp, and a value stored in three separate columns within a pandas data frame, e.g.:

I want to filter rows whose timestamp hours is equal to 23, and add a scalar to the values in the next column. How can I do this efficiently? The index column is not properly set in the dataset and I cannot rely on it.

Presently, I am using a for-loop to iterate over the rows, check if the hour in the timestamp is equal to 23, and modify the values in the corresponding cells, but it takes a lot of time. I tried to use the .groupby method suggested here as below, but that seems not to be working. It operates on the data two times, leaving the data unchanged and throwing SettingWithCopyWarning. Here is what I try. I am not sure if this is the best way to do it, though:

        for index, data_slice in df.groupby(df["Date"].dt.hour == 23):
            data_slice.loc["value"] += 1

okay got it, can you place the data as text please so we can copy that — anky, Apr 26 '19 at 17:43

score 2 · Accepted Answer · answered Apr 26 '19 at 17:44

2

Why groupby, you can try:

df.loc[df['Date'].dt.hour==23, 'value'] += 1

answered Apr 26 '19 at 17:44

Quang Hoang

146,074
10
56
74

This is so simple and brilliant. Thank you! – mfaieghi Apr 26 '19 at 17:52

Modify pandas dataframe cells by filtering datatime details

1 Answers1