Pandas dropna on specific rows

Question

import pandas as pd
df = pd.DataFrame({'ticker':['x','x','y','z','z'],
                   'bid':[1,2,np.nan,2,np.nan]})

Using pandas .dropna() is there anyway to drop the rows from a specified index range or subset of the data? For example in the DataFrame above, if I want to only drop rows in the index for where ticker equals 'z'. This would hopefully return:

ticker bid
x       1
x       2
y       np.nan
z       2

Possible duplicate of [How to drop rows of Pandas DataFrame whose value in a certain column is NaN](https://stackoverflow.com/questions/13413590/how-to-drop-rows-of-pandas-dataframe-whose-value-in-a-certain-column-is-nan) — Mayeul sgc, Oct 01 '19 at 15:59

ansev · Answer 1 · 2019-10-01T17:05:56.320

1

You can use dropna with mask and fillna:

df.mask(df.eq('z')).dropna(how='all').fillna({'ticker':'z'})

Output:

 ticker  bid
0      x  1.0
1      x  2.0
2      y  NaN
3      z  2.0

or

df.mask(df.eq('z')).dropna(how='all').mask(df.eq('z'),'z')

edited Oct 01 '19 at 17:05

answered Oct 01 '19 at 16:42

ansev

30,322
5
17
31

score 0 · Answer 2 · edited Oct 18 '19 at 16:57

0

One option is to just check the two conditions separately:

In [13]: df[(df['bid'].notnull()) | (df['ticker'] != 'z')]
Out[13]:

  ticker  bid
0      x  1.0
1      x  2.0
2      y  NaN
3      z  2.0

edited Oct 18 '19 at 16:57

ansev

30,322
5
17
31

answered Oct 01 '19 at 16:00

Randy

14,349
2
36
42

score 0 · Answer 3 · edited Oct 01 '19 at 16:13

0

df.loc[df.ticker == "z"] = df.loc[df.ticker == "z"].dropna()
df.dropna(subset=["ticker"])

Not sure if this is better for when I have more columns and need to specify if they have 2 missing nan's (using the .dropna(thresh=2) in the first drop)

edited Oct 01 '19 at 16:13

sophros

14,672
11
46
75

answered Oct 01 '19 at 16:08

Cr1064

409
5
15

Pandas dropna on specific rows

3 Answers3