7

if any elements are there along with nan then i want to keep element and want to delete nan only like

example 1 ->

index      values
0     [nan,'a',nan,nan]

output should be like

index   values

0         [a]

example 2->

index      values
0     [nan,'a',b,c]

1     [nan,nan,nan]

output should be like

index   values

0      [a,b,c]

1        []
Rakesh
  • 81,458
  • 17
  • 76
  • 113
Shubham Thagunna
  • 127
  • 1
  • 7

4 Answers4

8

This is one approach using df.apply.

import pandas as pd
import numpy as np

df = pd.DataFrame({"a": [[np.nan, np.nan, np.nan, "a", np.nan], [np.nan, np.nan], ["a", "b"]]})
df["a"] = df["a"].apply(lambda x: [i for i in x if str(i) != "nan"])
print(df)

Output:

        a
0     [a]
1      []
2  [a, b]
Rakesh
  • 81,458
  • 17
  • 76
  • 113
5

You can use the fact that np.nan == np.nan evaluates to False:

df = pd.DataFrame([[0, [np.nan, 'a', 'b', 'c']],
                   [1, [np.nan, np.nan, np.nan]],
                   [2, [np.nan, 'a', np.nan, np.nan]]],
                  columns=['index', 'values'])

df['values'] = df['values'].apply(lambda x: [i for i in x if i == i])

print(df)

   index     values
0      0  [a, b, c]
1      1         []
2      2        [a]

lambda is just an anonymous function. You could also use a named function:

def remove_nan(x):
    return [i for i in x if i == i]

df['values'] = df['values'].apply(remove_nan)

Related: Why is NaN not equal to NaN?

jpp
  • 159,742
  • 34
  • 281
  • 339
  • 1
    Thanks for the answer. Can you please tell me how lambda function is working here ? Difficult to understand i==i – Shubham Thagunna Jul 10 '18 at 10:44
  • @ShubhamThagunna, `lambda` is an anonymous function. It defines logic which is applied to each item in a sequence. Here the items are elements of the series `df['values']`. See update for the equivalent with a named function. – jpp Jul 10 '18 at 10:54
2
df['values'].apply(lambda v: pd.Series(v).dropna().values )
Pyd
  • 6,017
  • 18
  • 52
  • 109
0

You can use pd.Series.map on df.values

import pandas as pd
my_filter = lambda x: not pd.isna(x)
df['new_values'] = df['values'].map(lambda x: list(filter(my_filter, x)))
konvas
  • 14,126
  • 2
  • 40
  • 46