Pandas Dataframe Issue Converting Column dtype

Question

I have a simple pandas dataframe with a column:

col = [['A']]
data = [[1.0],[2.3],[3.4]]
df = pd.DataFrame.from_records(data, columns=col)

This creates a dataframe with one column of type np.float64, which is what I want.

Later in the process, I want to add another column of type string.

df['SOMETEXT'] = "SOME TEXT FOR ANALYSIS"

The dtype of this column is coming though as dtype of object, but I need it to be type string. So I do the following:

df['SOMETEXT'] = df['SOMETEXT'].astype(str)

If I look at the dtype again, I get the same dtype for that column: object.

I have a process down my workflow that is dtype sensitive and I need the column to be a string.

Any ideas?

array = df.to_records(index=False) # convert to numpy array

The dtypes on the array still carry the object dtype, but the columns should be a string.

Possible duplicate of [How to convert column with dtype as object to string in Pandas Dataframe](http://stackoverflow.com/questions/33957720/how-to-convert-column-with-dtype-as-object-to-string-in-pandas-dataframe) — languitar, Mar 14 '17 at 12:38

chrisaycock · Accepted Answer · 2017-03-14T12:52:04.903

3

In pandas, all strings are object type. It confused me too when I first started.

Once in NumPy, you can cast the string:

In [24]: array['SOMETEXT'].astype(str)
Out[24]: 
array(['SOME TEXT FOR ANALYSIS', 'SOME TEXT FOR ANALYSIS',
       'SOME TEXT FOR ANALYSIS'], 
      dtype='<U22')

edited Mar 14 '17 at 12:52

answered Mar 14 '17 at 12:37

chrisaycock

36,470
14
88
125

ya, but when you export the DF to an numpy array, it carries the object type and doesn't convert the data to a string. – code base 5000 Mar 14 '17 at 12:38
i forgot a piece of information, please see the updated question. – code base 5000 Mar 14 '17 at 12:39
That's just the internal implementation. Have a look at one of the answers of the potential duplicate for an explanation. – languitar Mar 14 '17 at 12:40
It won't work, as this is not intended. There is an explanation why this is of type object for strings. – languitar Mar 14 '17 at 16:31

Pandas Dataframe Issue Converting Column dtype

1 Answers1