3

I have an unparsed column in a dask dataframe (df) that I am using pandas to convert to datetime and put into a new column in the dask dataframe. However it breaks as column assignment doesn't support type DatetimeIndex. 

df['New Column'] = pd.to_datetime(np.array(df.index.values), format='%Y/%m/%d %H:%M')
Usherwood
  • 359
  • 3
  • 11
  • I suggest you to check this discussion on [github](https://github.com/dask/dask/issues/3717) too – rpanai Jul 19 '18 at 16:07

1 Answers1

3

this should work

import dask.dataframe as dd
# note df is a dask dataframe 
df['New Column'] = dd.to_datetime(df.index, format='%Y/%m/%d %H:%M')
moshevi
  • 4,999
  • 5
  • 33
  • 50
  • How did I not try this! Thank you – Usherwood Jul 19 '18 at 12:21
  • I can't find this function on the latest Dask dataframe [page](http://docs.dask.org/en/latest/dataframe-api.html)? Trying to convert `pd.to_datetime(firstdate, format='%Y-%m')` to its equivalent in Dask – shanlodh Apr 26 '19 at 06:30
  • It is [here](https://docs.dask.org/en/latest/dataframe-api.html#dask.dataframe.to_datetime) now in the docs for Dask 2.5.2. – edesz Oct 16 '19 at 01:13