Pandas: Every value of cell list to lower case

Question

I have a dataframe like this

# initialize list of lists 
data = [[1, ['ABC', 'pqr']], [2, ['abc', 'XY']], [3, np.nan]] 

# Create the pandas DataFrame 
data = pd.DataFrame(data, columns = ['Name', 'Val'])
data

    Name  Val
0   1     [ABC, pqr]
1   2     [abc, XY]
2   3     NaN

I am trying to convert every value in the list, to it's lower case

data['Val'] = data['Val'].apply(lambda x: np.nan if len(x) == 0 else [item.lower() for item in x])
data

However I get this error

TypeError: object of type 'float' has no len()

Expected final output

     Name  Val
0   1     [abc, pqr]
1   2     [abc, xy]
2   3     NaN

You have to escape the `NaN`, have a look at [this post](https://stackoverflow.com/questions/44061607/pandas-lambda-function-with-nan-support). — Max, May 05 '20 at 07:44
You can use `lambda x: np.nan if x is np.NaN else [item.lower() for item in x]`. — Henry Yik, May 05 '20 at 07:44
Henry, your answer is correct, would appreciate if you can write that as the answer — Hardik Gupta, May 05 '20 at 07:48
I am also getting this error - `TypeError: 'float' object is not iterable ` — Hardik Gupta, May 05 '20 at 07:52

jezrael · Accepted Answer · 2020-05-05T07:47:20.590

1

First idea is filter rows without missing values and processing:

m = data['Val'].notna()
data.loc[m, 'Val'] = data.loc[m, 'Val'].apply(lambda x: [item.lower() for item in x])
print (data)
   Name         Val
0     1  [abc, pqr]
1     2   [abc, xy]
2     3         NaN

Or you can processing only lists filtered by isinstance:

f = lambda x: [item.lower() for item in x] if isinstance(x, list) else np.nan
data['Val'] = data['Val'].apply(f)
print (data)
   Name         Val
0     1  [abc, pqr]
1     2   [abc, xy]
2     3         NaN

edited May 05 '20 at 07:47

answered May 05 '20 at 07:41

jezrael

822,522
95
1,334
1,252

I am getting this error also - `TypeError: 'float' object is not iterable ` – Hardik Gupta May 05 '20 at 07:52
@HardikGupta - Problem is with sample data? – jezrael May 05 '20 at 07:53
problem is with my data, not with sample data, I am unable to reproduce – Hardik Gupta May 05 '20 at 07:54
the list that I have, it is identifying it as float, and not list – Hardik Gupta May 05 '20 at 07:54
@HardikGupta - So need change `f = lambda x: [item.lower() for item in x] if isinstance(x, list) else np.nan` to `f = lambda x: [item.lower() for item in x] if not isinstance(x, float) else np.nan` ? – jezrael May 05 '20 at 07:55
yes this one worked, what is the issue here, how can I convert the list to be indentified as list itself – Hardik Gupta May 05 '20 at 07:57
@HardikGupta - Give me a sec. – jezrael May 05 '20 at 07:57
@HardikGupta - Can you try `import ast` and then `m = data['Val'].notna() data.loc[m, 'Val'] = data.loc[m, 'Val'].apply(lambda x: [item.lower() for item in ast.literal_eval(x)]) ` ? – jezrael May 05 '20 at 08:00
I get this error `ValueError: malformed node or string: ['some..value']` – Hardik Gupta May 05 '20 at 08:04
@HardikGupta - So it means wrong saved data? How should be processed? is possible replace `...` to emty string ? – jezrael May 05 '20 at 08:06

Pandas: Every value of cell list to lower case

1 Answers1