pandas: aggregate to keep first non-NaN value

Question

I have a dataframe with test_num as index:

file_num    6    7
test_num          
79        NaN    ↑
148         ↑  NaN

I need to reduce it to keep a first available direction (arrow) for any file_num:

           direction
test_num          
79          ↑
148         ↑

I've tried this:

fd.agg(lambda x: [a for a in x if a][0], axis=1)

test_num
79     NaN
148      ↑

fd.agg(lambda x: [a for a in x if a != pd.np.nan][0], axis=1)

test_num
79     NaN
148      ↑

How to do what I need here?

df.bfill() in the duplicated answer is genius. Here's what I was typing in when this was closed, which you might need in case you have to modify the specification going forward: `df.T.apply(lambda x: x.dropna().head(1)).fillna('').sum()` — user1717828, Aug 22 '18 at 18:16

score 2 · Accepted Answer · answered Aug 22 '18 at 17:55

You can use a uniform array in a groupby

df.groupby([*'A'*len(df.columns)], 1).first()

          A
test_num   
79        ↑
148       ↑

Or use a callable to the same effect

df.groupby(lambda x: 'A', 1).first()

          A
test_num   
79        ↑
148       ↑

score 1 · Answer 2 · answered Aug 22 '18 at 17:57

1

X = X.apply(lambda x: pd.Series(x.dropna().values), axis = 1)

        1.0
test_num    
79        ↑
148       ↑

answered Aug 22 '18 at 17:57

Raunaq Jain

917
7
13

pandas: aggregate to keep first non-NaN value

2 Answers2