Python pandas - Convert string to datetime without a year using pandas.to_datetime()

Question

I have a column in a dataframe which has timestamps and their datatype is object (string):

data_log = pd.read_csv(DATA_LOG_PATH)
print(data_log['LocalTime'])

0      09:38:49 
1      09:38:50 
2      09:38:51 
3      09:38:52 
4      09:38:53
         ...    
Name: LocalTime, Length: 872, dtype: object

Now I try to convert to datetime:

data_log['LocalTime'] = pd.to_datetime(data_log['LocalTime'], format='%H:%M:%S')
print(data_log['LocalTime'])

0     1900-01-01 09:38:49
1     1900-01-01 09:38:50
2     1900-01-01 09:38:51
3     1900-01-01 09:38:52
4     1900-01-01 09:38:53
              ...        
Name: LocalTime, Length: 872, dtype: datetime64[ns]

How do I remove that date there? I just want the time in the format that I specified, but it adds the 1900-01-01 to every row.

score 2 · Accepted Answer · answered Mar 21 '21 at 12:56

2

You can get the time part of a datetime series with Series.dt.time

print(data_log['LocalTime'].dt.time)

This series will consist of Python standard library datetime.time objects.

answered Mar 21 '21 at 12:56

Håken Lid

22,318
9
52
67

Thanks, I'm a big confused as to why it still says object as dtype but when I iterate through it and use the python type() function it does say datetime.time. Either way, thanks! – hawruh Mar 21 '21 at 13:13
1

Dtype object means that the Series is an array containing pointers to python objects. The objects don't even have to be of the same type. Other dtypes are much stricter and can only be used for a single type of data. That means that they can be more memory efficient and vectorized operations can be significantly faster, at the cost of flexibility. The object dtype is a used for python types that has no corresponding numpy type. https://stackoverflow.com/questions/29877508/what-does-dtype-object-mean-while-creating-a-numpy-array – Håken Lid Mar 21 '21 at 15:11

score 1 · Answer 2 · answered Mar 21 '21 at 12:52

You can do it in different ways from the datatype with 1900-01-01:

data_log['LocalTime'] = pd.Series([lt.time() for lt in data_log['LocalTime']])

or using a lambda function:

data_log['LocalTime'] = data_log.LocalTime.apply(lambda x: x.time(), axis = 1)

score 0 · Answer 3 · answered Mar 21 '21 at 12:54

For check the type in specific columns:

print(df['LocalTime'].dtypes)

to_dateTime func from pandas

https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html

df['LocalTime'] = pd.to_datetime(df['timestamp'], unit='s')

where: unit='s' defines the unit of the timestamp (seconds in this case)

For taking consider timezones:

df.rimestamp.dt.tz_localize('UTC').dt.tz_convert('Europe/Brussels')

Python pandas - Convert string to datetime without a year using pandas.to_datetime()

3 Answers3