2

Example df of how my data is stored:

df = pd.DataFrame({'DOB': {0: '2003-01-01 00:00:00+00', 1: '2003-01-02 00:00:00+00'}})

I want to convert the column to datetime, and only keep the Year-Month-Day values. Basically, exactly as this question:

Removing the timestamp from a datetime in pandas dataframe

and this one:

Convert column of date objects in Pandas DataFrame to strings

Except using these answer, extracting the dates from the column which is already a Datetime format converts the column to an Object format. This is also discussed in the comments, but seems to have been moved to chat which isn't available anymore.

Result in first answer:

[in] pd.to_datetime(df['DOB']).dt.date
[out]
0    2003-01-01
1    2003-01-02
Name: DOB, dtype: object

Result of second answer:

[in] pd.to_datetime(df['DOB']).dt.strftime('%Y-%m-%d')
[out] 
0    2003-01-01
1    2003-01-02
Name: DOB, dtype: object

As you can see, it's not a Datetime format anymore but an object. Then I can't use it furthermore in my script.

Using the format = "%Y-%m-%d option of pd.to_datetime doesn't work too, as the timestamp is still present:

[in] pd.to_datetime(df['DOB'], format = "%Y-%m-%d")
[out] 
0   2003-01-01 00:00:00+00:00
1   2003-01-02 00:00:00+00:00
Name: DOB, dtype: datetime64[ns, UTC]

What is a simple solution to make sure the column stays in a datetime format, while only keeping the date as format?

B.Quaink
  • 456
  • 1
  • 4
  • 18
  • what type do you want? object doesn't mean anything specific – mozway Feb 16 '22 at 11:17
  • I want the column to be in datetime format. Like the bottom section of code is showing, but without the timestamp. – B.Quaink Feb 16 '22 at 11:18
  • Then what is wrong with `pd.to_datetime(df['DOB']).dt.date`? This will give you a `datetime.date` – mozway Feb 16 '22 at 11:20
  • It doesn't give a datetime, it returns as ```dtype: object``` . That's the 1st answers issue. – B.Quaink Feb 16 '22 at 11:21
  • This is just a representation, the real type is `datetime.date`: see [here](https://stackoverflow.com/a/71141043/16343464). You actually have the same thing with many other types – mozway Feb 16 '22 at 11:32
  • I'm getting constant issues when performing simple functions using datetime on this column though this way, and not when it's a datetime format. – B.Quaink Feb 16 '22 at 11:35
  • Maybe this is the real question you should ask ;) – mozway Feb 16 '22 at 11:37
  • Maybe it is! But all of my issues were solved once this column is just a datetime, and removes later on a lot of code dealing with this issue. That's why I was hoping there was a solution before dealing with it. – B.Quaink Feb 16 '22 at 11:38
  • Note that there is a difference between the datetime type pandas uses and native Python datetime. While Python has a date type, pandas does not. With pandas datetime you always have date ***and*** time. If you switch to native Python's date type (`.dt.date`), you loose a lot of functionality. – FObersteiner Feb 16 '22 at 11:42
  • Yes the types are different (this is why pandas displays 'object' as it is not native). I guess this might be the issue, but for that a clear example should be provided (maybe in a fresh question) – mozway Feb 16 '22 at 11:42

2 Answers2

5

IMO there is no issue here:

s = pd.to_datetime(pd.Series(['2021-02-01 00:00:00']))
s
# 0   2021-02-01
# dtype: datetime64[ns]

And indeed, the displayed type is "object":

s.dt.date
# 0    2021-02-01
# dtype: object

But this doesn't mean much, the type is really datetime.date:

type(s.dt.date[0])
# datetime.date
mozway
  • 194,879
  • 13
  • 39
  • 75
  • I'm getting constant issues when performing simple functions using datetime on this column though this way, and not when it's a datetime format. – B.Quaink Feb 16 '22 at 11:35
0

I have it working with two extra lines, but was hoping there would be a simpler solution to this issue:

df['DOB'] = pd.to_datetime(df['DOB'])
df['DOB'] = DF['DOB'].dt.date
df['DOB'] = pd.to_datetime(df['DOB'])

# gives
[in] df
[out]
0    2003-01-01
1    2003-01-02
Name: DOB, dtype: datetime64[ns]
B.Quaink
  • 456
  • 1
  • 4
  • 18