Rounding time off to the nearest second - Python

Question

I have a large dataset with more than 500 000 date & time stamps that look like this:

date        time
2017-06-25 00:31:53.993
2017-06-25 00:32:31.224
2017-06-25 00:33:11.223
2017-06-25 00:33:53.876
2017-06-25 00:34:31.219
2017-06-25 00:35:12.634

How do I round these timestamps off to the nearest second?

My code looks like this:

readcsv = pd.read_csv(filename)
log_date = readcsv.date
log_time = readcsv.time

readcsv['date'] = pd.to_datetime(readcsv['date']).dt.date
readcsv['time'] = pd.to_datetime(readcsv['time']).dt.time
timestamp = [datetime.datetime.combine(log_date[i],log_time[i]) for i in range(len(log_date))]

So now I have combined the dates and times into a list of datetime.datetime objects that looks like this:

datetime.datetime(2017,6,25,00,31,53,993000)
datetime.datetime(2017,6,25,00,32,31,224000)
datetime.datetime(2017,6,25,00,33,11,223000)
datetime.datetime(2017,6,25,00,33,53,876000)
datetime.datetime(2017,6,25,00,34,31,219000)
datetime.datetime(2017,6,25,00,35,12,634000)

Where do I go from here? The df.timestamp.dt.round('1s') function doesn't seem to be working? Also when using .split() I was having issues when the seconds and minutes exceeded 59

Many thanks

https://stackoverflow.com/questions/3463930/how-to-round-the-minute-of-a-datetime-object-python/10854034#10854034 use 1*60 as a parameter. — Borko Kovacev, Dec 13 '17 at 11:49
What is your dataset made of? A pandas dataframe, a CSV file? Is there anything you have tried to solve the problem? — Nils Werner, Dec 13 '17 at 11:51
Does something like this help? `round(float('2017-06-25 00:31:53.993'.split()[1].split(':')[-1]))` — srikavineehari, Dec 13 '17 at 12:14
The data is in a CSV file and Iv used pandas to get the timestamps into this format. @srig That seems to be working thank you. — Jetman, Dec 13 '17 at 12:21
@Jetman If you're using pandas, there are much better solutions. Please take a look at my answer, and accept it if it helps you. Thanks. — cs95, Dec 13 '17 at 14:22

score 32 · Answer 1 · edited Apr 08 '21 at 18:18

32

Without any extra packages, a datetime object can be rounded to the nearest second with the following simple function:

import datetime as dt

def round_seconds(obj: dt.datetime) -> dt.datetime:
    if obj.microsecond >= 500_000:
        obj += dt.timedelta(seconds=1)
    return obj.replace(microsecond=0)

edited Apr 08 '21 at 18:18

Erik Kalkoken

30,467
8
79
114

answered Mar 15 '18 at 21:52

derpedy-doo

1,097
1
18
22

4

Small improvement here: To satisfy python's naming conventions, rename `roundSeconds` to `round_seconds` and all other camelCase names accordingly. Other than that, great answer! – Don Fruendo Dec 14 '20 at 14:46
This function leaves small amount of microseconds due to float precision. Answer by @Maciejo95 works without this problem. – Karol Zlot Mar 12 '21 at 07:04
Careful: This function is mutating the given object. – Erik Kalkoken Apr 06 '21 at 18:36
@ErikKalkoken How is this mutating the given object? In a quick test I don't see `.second` or `.microsecond` modified on the given object. – derpedy-doo Apr 06 '21 at 21:22
@qewg in what situations would you experience some microseconds left over? I've seen `microseconds` correctly set to `0` in all my cases. – derpedy-doo Apr 06 '21 at 21:24
1

@electrovir I was wrong. The function is indeed not mutating the object. The way the function is written tripped me a bit, because `new_date_time = date_time_object` does not create a copy. You can do all operations on the given object and it will still work. That is because datetime objects are immutable. – Erik Kalkoken Apr 07 '21 at 11:59
1

@ErikKalkoken gotcha, that makes sense. Feel free to edit the answer to make it read better! Python (along with all its conventions) isn't my one of my strongest languages. – derpedy-doo Apr 07 '21 at 17:57
I do not like not commented magic numbers in the code. Not going to give a try to this solution. – Sany Jun 07 '23 at 06:49

score 18 · Answer 2 · edited Oct 31 '19 at 22:34

18

The question doesn't say how you want to round. Rounding down would often be appropriate for a time function. This is not statistics.

rounded_down_datetime = raw_datetime.replace(microsecond=0)

edited Oct 31 '19 at 22:34

jonrsharpe

115,751
26
228
437

answered Oct 16 '19 at 13:56

mike rodent

14,126
11
103
157

1

Rounding to seconds means normally for >=0.5 you round to 1 for <0.5 you round to 0. Your answer is valid for cutting decimal part. It is not rounding. – Sany Jun 07 '23 at 07:27

score 14 · Answer 3 · answered Apr 15 '20 at 09:22

14

If anyone wants to round a single datetime item off to the nearest second, this one works just fine:

pandas.to_datetime(your_datetime_item).round('1s')

answered Apr 15 '20 at 09:22

Maciejo95

159
1
6

Also works with pandas column data. I feel this is the best answer. – TPM Sep 22 '21 at 11:16
1

this solution changes output type (datetime.datetime -> pd.Timestamp), what not always is desirabled – Daniel R Dec 10 '21 at 10:12

score 9 · Answer 4 · answered Dec 13 '17 at 14:22

If you're using pandas, you can just round the data to the nearest second using dt.round -

df

                timestamp
0 2017-06-25 00:31:53.993
1 2017-06-25 00:32:31.224
2 2017-06-25 00:33:11.223
3 2017-06-25 00:33:53.876
4 2017-06-25 00:34:31.219
5 2017-06-25 00:35:12.634

df.timestamp.dt.round('1s')

0   2017-06-25 00:31:54
1   2017-06-25 00:32:31
2   2017-06-25 00:33:11
3   2017-06-25 00:33:54
4   2017-06-25 00:34:31
5   2017-06-25 00:35:13
Name: timestamp, dtype: datetime64[ns]

If timestamp isn't a datetime column, convert it first, using pd.to_datetime -

df.timestamp = pd.to_datetime(df.timestamp)

Then, dt.round should work.

Thanks very much, I can't believe I didn't think about that. — Jetman, Dec 14 '17 at 05:23

score 5 · Answer 5 · answered Mar 18 '20 at 15:45

5

Alternate version of @electrovir 's solution:

import datetime

def roundSeconds(dateTimeObject):
    newDateTime = dateTimeObject + datetime.timedelta(seconds=.5)
    return newDateTime.replace(microsecond=0)

answered Mar 18 '20 at 15:45

gerardw

5,822
46
39

srikavineehari · Accepted Answer · 2017-12-13T12:59:04.297

Using for loop and str.split():

dts = ['2017-06-25 00:31:53.993',
       '2017-06-25 00:32:31.224',
       '2017-06-25 00:33:11.223',
       '2017-06-25 00:33:53.876',
       '2017-06-25 00:34:31.219',
       '2017-06-25 00:35:12.634']

for item in dts:
    date = item.split()[0]
    h, m, s = [item.split()[1].split(':')[0],
               item.split()[1].split(':')[1],
               str(round(float(item.split()[1].split(':')[-1])))]

    print(date + ' ' + h + ':' + m + ':' + s)

2017-06-25 00:31:54
2017-06-25 00:32:31
2017-06-25 00:33:11
2017-06-25 00:33:54
2017-06-25 00:34:31
2017-06-25 00:35:13
>>>

You could turn that into a function:

def round_seconds(dts):
    result = []
    for item in dts:
        date = item.split()[0]
        h, m, s = [item.split()[1].split(':')[0],
                   item.split()[1].split(':')[1],
                   str(round(float(item.split()[1].split(':')[-1])))]
        result.append(date + ' ' + h + ':' + m + ':' + s)

    return result

Testing the function:

dts = ['2017-06-25 00:31:53.993',
       '2017-06-25 00:32:31.224',
       '2017-06-25 00:33:11.223',
       '2017-06-25 00:33:53.876',
       '2017-06-25 00:34:31.219',
       '2017-06-25 00:35:12.634']

from pprint import pprint

pprint(round_seconds(dts))

['2017-06-25 00:31:54',
 '2017-06-25 00:32:31',
 '2017-06-25 00:33:11',
 '2017-06-25 00:33:54',
 '2017-06-25 00:34:31',
 '2017-06-25 00:35:13']
>>>

Since you seem to be using Python 2.7, to drop any trailing zeros, you may need to change:

str(round(float(item.split()[1].split(':')[-1])))

to

str(round(float(item.split()[1].split(':')[-1]))).rstrip('0').rstrip('.')

I've just tried the function with Python 2.7 at repl.it and it ran as expected.

I believe this would fail for the edge case `'2017-06-25 00:31:59.993'`. — Serge Mosin, Jun 13 '18 at 14:52
This only works when rounding down. Rounding up seconds doesn't update the minutes. — bgusach, Jul 26 '21 at 08:26

Rudradev Pal · Answer 7 · 2017-12-13T13:15:53.177

If you are storing dataset into a file you can do like this:

with open('../dataset.txt') as fp:
    line = fp.readline()
    cnt = 1
    while line:
        line = fp.readline()
        print "\n" + line.strip()
        sec = line[line.rfind(':') + 1:len(line)]
        rounded_num = int(round(float(sec)))
        print line[0:line.rfind(':') + 1] + str(rounded_num)
        print abs(float(sec) - rounded_num)
        cnt += 1

If you are storing dataset in a list:

dts = ['2017-06-25 00:31:53.993',
   '2017-06-25 00:32:31.224',
   '2017-06-25 00:33:11.223',
   '2017-06-25 00:33:53.876',
   '2017-06-25 00:34:31.219',
   '2017-06-25 00:35:12.634']

for i in dts:
    line = i
    print "\n" + line.strip()
    sec = line[line.rfind(':') + 1:len(line)]
    rounded_num = int(round(float(sec)))
    print line[0:line.rfind(':') + 1] + str(rounded_num)
    print abs(float(sec) - rounded_num)

score 1 · Answer 8 · answered Apr 11 '21 at 18:37

Another way of doing this that:

doesn't involve string manipulation
uses Python's built-in round
doesn't mutate the original timedelta, but gives a new one
is a one liner :)

import datetime

original = datetime.timedelta(seconds=50, milliseconds=20)
rounded = datetime.timedelta(seconds=round(original.total_seconds()))

score 1 · Answer 9 · answered Jul 26 '21 at 08:53

Here's a simple solution that properly rounds up and down and doesn't use any string hacks:

from datetime import datetime, timedelta

def round_to_secs(dt: datetime) -> datetime:
    extra_sec = round(dt.microsecond / 10 ** 6)
    return dt.replace(microsecond=0) + timedelta(seconds=extra_sec)

Some examples:

now = datetime.now()
print(now)                 # 2021-07-26 10:43:54.397538
print(round_to_secs(now))  # 2021-07-26 10:43:54 -- rounded down

now = datetime.now()
print(now)                 # 2021-07-26 10:44:59.787438
print(round_to_secs(now))  # 2021-07-26 10:45:00  -- rounded up taking into account secs and minutes

score 0 · Answer 10 · answered Apr 20 '20 at 09:50

An elegant solution that only requires the standard datetime module.

import datetime

            currentimemili = datetime.datetime.now()
            currenttimesecs = currentimemili - \
                datetime.timedelta(microseconds=currentimemili.microsecond)
            print(currenttimesecs)

score 0 · Answer 11 · answered Jul 16 '20 at 00:52

I needed it, so I adjusted @srisaila to work for 60 sec/mins. Horribly complicated style, but basic functions.

def round_seconds(dts):
    result = []
    for item in dts:
        date = item.split()[0]
        h, m, s = [item.split()[1].split(':')[0],
                   item.split()[1].split(':')[1],
                   str(round(float(item.split()[1].split(':')[-1])))]
        if len(s) == 1:
            s = '0'+s
        if int(s) == 60:
            m_tmp = int(m)
            m_tmp += 1
            m = str(m_tmp)
            if(len(m)) == 1:
                m = '0'+ m
            s = '00'
        if m == 60:
            h_tmp = int(h)
            h_tmp += 1
            h = str(h_tmp)
            if(len(h)) == 1:
                print(h)
                h = '0'+ h
            m = '00'
        result.append(date + ' ' + h + ':' + m + ':' + s)
    return result

score -1 · Answer 12 · edited Jun 01 '22 at 21:31

-1

def round_sec(dt):
   return dt.replace(microsecond=0)

edited Jun 01 '22 at 21:31

Dharman

30,962
25
85
135

answered Jun 01 '22 at 10:29

Savun Cheam

11
1

Rounding time off to the nearest second - Python

12 Answers12

Linked