Is there a faster way to do this loop?

Question

I want to create a new column using the following loop. The table just has the columns 'open', and 'start'. I want to create a new column 'startopen', where if 'start' equals 1, then 'startopen' is equal to 'open'. Otherwise, 'startopen' is equal to whatever 'startopen' was in the row above of this newly created column. Currently I'm able to achieve this using the following:

for i in range(df.shape[0]):
    if df['start'].iloc[i] == 1:
        df.loc[df.index[i],'startopen'] = df.loc[df.index[i],'open']
    else:
        df.loc[df.index[i],'startopen'] = df.loc[df.index[i-1],'startopen']

This works, but is very slow for large datasets. Are there any built in functions that can do this faster?

https://stackoverflow.com/q/20109391/4046632 – buran May 28 '22 at 05:49 — buran, May 28 '22 at 05:49

score 2 · Answer 1 · answered May 28 '22 at 05:52

2

I want to create a new column 'startopen', where if 'start' equals 1, then 'startopen' is equal to 'open'

Otherwise, 'startopen' is equal to whatever 'startopen' was in the row above of this newly created column.

IIUC, otherwise part is equal to forward fill the not 1 startopen with last equal 1 startopen

df['startopen'] = pd.Series(np.where(df['start'].eq(1), df['open'], np.nan), index=df.index).ffill()

answered May 28 '22 at 05:52

Ynjxsjmh

28,441
6
34
52

1

I like the method :p – mozway May 28 '22 at 06:24
Thank you! Can the where function be used to reference itself? For example, if I were to want it to first reference 'start', and if not equal to 1, then reference the preceding row in 'startopen' and perform actions based on that? – batataman May 28 '22 at 06:29
@batataman It's out of my knowledge. – Ynjxsjmh May 28 '22 at 06:43

Is there a faster way to do this loop?

1 Answers1