0

I am trying to convert NaN (dtype: float64) values in a Pandas DataFrame column into NaT values.

Please note that I have several DataFrames with the same Order_date column. Some Order_date columns' dtypes are float64 (filled with NaN) while others' dtypes are datetime64[ns] (filled with NaT).

I tried the following: 

df.loc[:,'Order_date'] = df.loc[:,'Order_date'].astype(np.datetime64).fillna(pd.NaT)

However, I get an Error result: 

TypeError: cannot astype a datatimelike from [datetime64[ns]] to [datetime64].

What is reason behind this error? I think the error is due to how several Order_date columns in some DataFrames have NaT values (datetime64[ns]). What can I do to successfully convert only the Order_date columns with NaN values into NaT values in Pandas and leave the remaining Order_date columns that already have NaT values as is?

CPU
  • 267
  • 1
  • 6
  • 16
  • Possible duplicate of [How to convert Pandas DataFrame column from Pandas datetime64\[ns\] to Pyodbc SQL\_Timestamp](https://stackoverflow.com/questions/47643804/how-to-convert-pandas-dataframe-column-from-pandas-datetime64ns-to-pyodbc-sql) – Gord Thompson Dec 06 '17 at 16:29
  • 1
    This is not a duplicate question. This question is asking how to convert NaN to NaT values. – CPU Dec 06 '17 at 16:41

2 Answers2

0

I might be misunderstanding the requirements of the problem, but wouldn't

df['Order_date'].fillna(pd.NaT, inplace=True)

do the job?

Orest Xherija
  • 434
  • 4
  • 18
  • I tried your solution, but it does not work. It converts NaN values to _None_ values. I get the following error: pyodbc.DataError: **('22008', '[22008] [Microsoft][ODBC Microsoft Access Driver]Datetime field overflow (SQLExecDirectW)')** – CPU Dec 06 '17 at 13:28
  • If you can add a small snippet of your dataframe that illustrates your point, I can probably find a way around the problem. I am afraid I am misunderstanding the problem at the moment. – Orest Xherija Dec 07 '17 at 04:19
0

What about using pd.to_datetime()? Like so:

pd.to_datetime(df.loc[:,'Order_date'])

Should return pd.NaT instead of np.NaN

Ian Thompson
  • 2,914
  • 2
  • 18
  • 31