Remove NaN values from certain columns Pandas Series

Question

I have the following DF:

AA   BB  CC 
1    1   1
NaN  3   NaN
4    4   6
NaN  NaN 3
         NaN
         NaN
     4

The output should be:

AA   BB  CC 
1     1   1
4     3   6
      4   3
          4

I've tried:

df = df.dropna(subset=['AA', 'BB', 'CC'])
     AA      BB CC
0    2       3  1
2    5       5  6

and this is the output I get.

Is there anything else I should be doing differently?

[This](https://stackoverflow.com/questions/44558215/python-justifying-numpy-array) should be helpful. — Quang Hoang, Apr 06 '21 at 04:13
Maybe you can do `df.apply(lambda x: pd.Series(x.dropna().values)).fillna('')` — Joe Ferndz, Apr 06 '21 at 04:19
.You can do this in `pandas` by: `resultdf=df.ffill().apply(lambda x:x.drop_duplicates().reset_index().drop(columns='index').squeeze())` — Anurag Dabas, Apr 06 '21 at 04:25

score 1 · Accepted Answer · answered Apr 06 '21 at 04:24

1

You can use:

df.apply(lambda x: x.dropna().reset_index(drop = True))

    AA  BB  CC
0   1.0 1.0 1.0
1   4.0 3.0 6.0
2   NaN 4.0 3.0
3   NaN NaN 4.0

answered Apr 06 '21 at 04:24

Pygirl

Really appreciate your input here and this would work, but I still end up with NaN values, Would there be a way to not have Pandas add NaN when a value is missing and other columns are larger? The whole point of what I am looking to do is to process data without any NULL values. This is the result I am looking for I just need to remove the remaining NaN values as well. – ApacheOne Apr 06 '21 at 04:46
@Aldo: use `df.apply(lambda x: x.dropna().reset_index(drop = True)).fillna('')` --> just add `fillna('')` – Pygirl Apr 06 '21 at 05:01

1 Answers1