11

I want to change dtype of one data frame column (from datetime64 to object).

First of all, I create data frame:

Python 2.6.8 (unknown, Jan 26 2013, 14:35:25) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> values = pd.Series(i for i in range(5))
>>> dates = pd.date_range('20130101',periods=5)
>>> df = pd.DataFrame({'values': values, 'dates': dates})
>>> df
/usr/local/lib/python2.6/dist-packages/pandas/core/config.py:570: DeprecationWarning: height has been deprecated.

  warnings.warn(d.msg, DeprecationWarning)
                dates  values
0 2013-01-01 00:00:00       0
1 2013-01-02 00:00:00       1
2 2013-01-03 00:00:00       2
3 2013-01-04 00:00:00       3
4 2013-01-05 00:00:00       4

It have two columns: one is datetime64 and other one is int64 dtype:

>>> df.dtypes
dates     datetime64[ns]
values             int64
dtype: object

In pandas documentation I found how to convert series to any dtypes. It looks like what I need:

>>> df['dates'].astype(object)
0    2013-01-01 00:00:00
1    2013-01-02 00:00:00
2    2013-01-03 00:00:00
3    2013-01-04 00:00:00
4    2013-01-05 00:00:00
Name: dates, dtype: object

But when I assign this series as dataframe column, I got a datetime64 dtype again. 

>>> df['dates'] = df['dates'].astype(object)
>>> df.dtypes
dates     datetime64[ns]
values             int64
dtype: object

Please, help. How to convert data frame's column to object dtype? Thanks.

ghostishev
  • 143
  • 1
  • 2
  • 8
  • why are you trying to do this? object representation in the case of a datetime is very inefficient so pandas converts internally. – Jeff Oct 18 '13 at 12:07
  • I want to convert datetime to object because pandas have a bug (see http://stackoverflow.com/questions/19300730) in pivot totals calculation if column (which used as header) type is datetime. – ghostishev Oct 18 '13 at 12:24
  • 3
    the solution in that issue looks fine, just transpose the rows/columns. What you are trying to do has nothing to do with ``datetime64[ns]`` as a dtype and will not help. – Jeff Oct 18 '13 at 12:38
  • yes, but only if pivot row column isn't datatime too. – ghostishev Oct 18 '13 at 13:12

4 Answers4

8

If you really want to change from datatype of datetime64[ns] to object, you could run something like this:

df['dates'] = df['dates'].apply(lambda x: str(x))
print df.types # Can verify to see that dates prints out as an object
Will
  • 11,276
  • 9
  • 68
  • 76
  • 1
    You don't need a lambda in your example, actually: `df['dates'] = df['dates'].apply(str)` would work and read the same. – varepsilon Dec 16 '20 at 17:08
  • This is the right answer. I encountered the same problem when converting a column of int64 to object by using .astype('object'). Printing out column details using .info() shows this column has been converted however the encoding failed saying this columns is not 'str' – oy321 Sep 09 '21 at 07:47
1

If you want to convert Date column which is object type to datetime64[ns] dtype;, then following code will work:

df['Date']=pd.to_datetime(df['Date'])
double-beep
  • 5,031
  • 17
  • 33
  • 41
Darshan Chheda
  • 23
  • 6
0

Is this what you are after?

In [9]: pd.pivot_table(data=df,rows='columns',cols='rows',values='values',margins=True).T
Out[9]: 
columns  2013-01-01 00:00:00  2013-01-02 00:00:00  2013-01-03 00:00:00  2013-01-04 00:00:00  2013-01-05 00:00:00       All
rows                                                                                                                      
a                          0                  NaN                    2                    3                  NaN  1.666667
b                        NaN                    1                  NaN                  NaN                    4  2.500000
All                        0                    1                    2                    3                    4  2.000000
Jeff
  • 125,376
  • 21
  • 220
  • 187
0

Not proficient with lambda usage. In some simple case, df['dates'].astype(str) would work also.

Note: it doesn't work when there are NaN in a column.

Not solution to OP, but others may find help in this question. Almost duplicate, but it's mostly talk about convert to numeric.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Shihe Zhang
  • 2,641
  • 5
  • 36
  • 57