loop under df.loc in python dataframe

Question

Currently, I have code where I am checking the last 5 slopes from dataframe:

    df = dfm.set_index(['token', 'serial']).unstack()
    dfs = (df.loc[
                df[('slope', 16)].gt(0)
                & df[('slope', 15)].gt(df[('slope', 16)])
                & df[('slope', 14)].gt(df[('slope', 15)])
                & df[('slope', 13)].gt(df[('slope', 14)])
                & df[('slope', 12)].gt(df[('slope', 13)])
                & df[('slope', 11)].gt(df[('slope', 12)])
             ]

                    .stack(level=1)
                    .reset_index()
                    .query('serial <= 1')
            )

It's small when we look for only till 16, but for testing, I need to check till any number like 80 / 100. So how can we loop instead of adding 100 lines?

    df = dfm.set_index(['token', 'serial']).unstack()
    dfs = (df.loc[
                df[('slope', 96)].gt(0)
                & df[('slope', 95)].gt(df[('slope', 96)])
                #............. from 2 to 95
                & df[('slope', 1)].gt(df[('slope', 2)])
             ]

                    .stack(level=1)
                    .reset_index()
                    .query('serial <= 1')
            )

blacksnail789521 · Answer 1 · 2022-08-25T14:02:15.223

Let's say you have three conditions:

c1 = df['slope'] == 1 # pd.Series
c2 = df['slope'] == 2
c3 = df['slope'] == 3

According to https://stackoverflow.com/a/20528566/3853182, you can use

np.logical_and.reduce([c1, c2, c3])

or

np.all([c1, c2, c3], axis = 0)

So, in your example, it should be something like this:

    df = dfm.set_index(['token', 'serial']).unstack()
    dfs = (df.loc[
                df[('slope', 96)].gt(0)
                & np.all([ df[('slope', i)].gt(df[('slope', i+1)]) for i in range(1, 96) ], axis = 0)
             ]

                    .stack(level=1)
                    .reset_index()
                    .query('serial <= 1')
            )

As you might expect, if you want to apply | on multiple conditions (or any array-like object), you can use

np.logical_or.reduce([c1, c2, c3])

or

np.any([c1, c2, c3], axis = 0)

loop under df.loc in python dataframe

1 Answers1