2

Suppose I have a column that contains object which contains float(like this "12.45"). How can I check if this column exists in panda data frame. I want to find this column first and convert all of it's values into float. In another words how can I look for the columns that have objects containing numeric values like floats "12.54". I do not want to look for the objects that contain non numeric values like "cat" or "dog"

for example:

import pandas as pd

df = pd.DataFrame({"column1":[ "12.44", "56.78", "45.87"],
    "column2":["cat", "dog", "horse"]})

I want to check if column1 exists in my columns in order to convert all of it's values to float

tdelaney
  • 73,364
  • 6
  • 83
  • 116
MZG
  • 327
  • 2
  • 12

2 Answers2

2

You can use pd.to_numeric to try to convert the strings to numeric values. Then, check for each element in the converted columns whether it is an instance of float type by using .applymap() and isinstance(x, float). Finally, check for any column value in a column is of type float by .any():

df.apply(pd.to_numeric, errors="ignore").applymap(lambda x: isinstance(x, float), na_action='ignore').any()

Result:

column1     True
column2    False
dtype: bool

True value of column1 corresponds to it has at least one element of type float

Actually, this solution can also check for individual elements whether of float type by removing the .any() at the end:

df.apply(pd.to_numeric, errors="ignore").applymap(lambda x: isinstance(x, float), na_action='ignore')

Result:

   column1  column2
0     True    False
1     True    False
2     True    False
SeaBean
  • 22,547
  • 3
  • 13
  • 25
0

You can try to convert column by column and catch errors along the way. Columns that won't convert are unchanged because the exception is raised before the column is reassigned.

import pandas as pd

df = pd.DataFrame({"column1":[ "12.44", "56.78", "45.87"],
    "column2":["cat", "dog", "horse"],
    "column3":["1", "2", "3"]})

for colname in df.columns:
    try:
        df[colname] = df[colname].astype('float')
        print(f"{colname} converted")
    except ValueError:
        print(f"{colname} failed")
    
print(df.dtypes)
tdelaney
  • 73,364
  • 6
  • 83
  • 116
  • thanks for the answer but for this line : df[colname] = df[colname].astype('float') I still get this error "could not convert string to float" – MZG Oct 01 '21 at 19:52
  • Running this exact script works for me. That is the text of the `ValueError` which the code catches and ignores. Are you running it some other way, not in an try/except block? – tdelaney Oct 01 '21 at 19:56
  • I tried it in try except block same as your answer but in debugger I got error – MZG Oct 01 '21 at 20:08
  • @MZG - that's likely because of the debugger. You should be able to continue and go through the exception handler. – tdelaney Oct 01 '21 at 23:23