14

What does python time and datetime module return on the leap second?

What will I get when we are at 23:59:60.5 if I call:

  • time.time()
  • datetime.datetime.utcnow()
  • datetime.datetime.now(pytz.utc)

Also, any difference between py2.7 and py3?

Why it is confusing (at least for me):

From the datetime docs I see:

Unlike the time module, the datetime module does not support leap seconds.

On the time docs I see there is "support" for leap seconds when parsing with strptime. But there is no comment about time.time().

I see that using time I get:

>>> time.mktime(time.strptime('2016-06-30T23:59:59', "%Y-%m-%dT%H:%M:%S"))
1467327599.0
>>> time.mktime(time.strptime('2016-06-30T23:59:60', "%Y-%m-%dT%H:%M:%S"))
1467327600.0
>>> time.mktime(time.strptime('2016-07-01T00:00:00', "%Y-%m-%dT%H:%M:%S"))
1467327600.0

And datetime just blows up:

>>> dt.datetime.strptime('2016-06-30T23:59:60', "%Y-%m-%dT%H:%M:%S")
Traceback (most recent call last):
  File "<stdin>", line 1, in &lt;module>
ValueError: second must be in 0..59

Then what will I get at that exact time (in the middle of the leap second)?

I have read about rubber times, clocks slowing down, repeating seconds, and all kind of crazy ideas, but what should I expect on python?

Note: In case you wonder if I don't have anything better to do that care about it, a leap second is approaching!!!!

Mario Corchero
  • 5,257
  • 5
  • 33
  • 59
  • http://stackoverflow.com/questions/21027639/python-datetime-not-accounting-for-leap-second-properly – Padraic Cunningham Sep 25 '16 at 12:01
  • 1
    I'd expect nothing to happen. Your *computer clock* won't honour the leap second; it'll just continue on until the next ntp clock sync, after which your computer clock will be correct again. – Martijn Pieters Sep 25 '16 at 12:01
  • @PadraicCunningham: all the datetime posts are about how to *represent* a leap second with datetime, which it can't. This is about what will happen when you try to handle time *at the exact leap second taking place*. Which, on current computers, means 'nothing' as the computer clock won't honour the leap second as there is no facility to tell your computer clock ahead of time to incorporate one. – Martijn Pieters Sep 25 '16 at 12:03
  • @MartijnPieters, it shows how to handle leap seconds so datetime won't *just blows up* – Padraic Cunningham Sep 25 '16 at 12:05
  • @MartijnPieters: Does that means that if I want to schedule to happen everyday exactly at 00:00 that day it will happen 1s before? As the computer will give me the day of tomorrow until it gets on sync? Because if so, that is kind of an issue. – Mario Corchero Sep 25 '16 at 12:05
  • 1
    If you want to have a datetime with leap second support, try astropy.time – MaxNoe Sep 25 '16 at 12:06
  • @PadraicCunningham: but it won't be asked to handle a leap second. The computer won't even know there is one in effect. – Martijn Pieters Sep 25 '16 at 12:12
  • 1
    @MarioCorchero: yes, it'll be 1 second early. – Martijn Pieters Sep 25 '16 at 12:13
  • @MarioCorchero: this has happened 26 times already. If you need events to happen with *that level of precision* you shouldn't be using Python and your standard computer clock anyway. – Martijn Pieters Sep 25 '16 at 12:16
  • 1
    @MartijnPieters: *that level of precision* is a second, what is wrong with python? I can agree about not using the computer clock, but I don't see the issue with python. – Mario Corchero Sep 25 '16 at 12:20
  • 3
    @MarioCorchero: Python is not a real-time programming language. Don't use it when timing precision is so important that leap seconds can make a difference. Even your general use *Operating System* (Windows, OS X, Linux, etc.) won't be able to make such guarantees. That's what a [Real Time OS](https://en.wikipedia.org/wiki/Real-time_operating_system) is for. – Martijn Pieters Sep 25 '16 at 12:25
  • 2
    Just thought I'd point out there are plenty of use cases for this level of precision outside a real-time system. In general, if you're dealing with data generated by such a system, and it's is regrettably in UTC (instead of say, TAI), you need to think about leap-seconds. This is particularly important, since it's the one of the few problems that just converting everything to Unix time doesn't solve. – morganwahl Jan 08 '21 at 03:46

2 Answers2

12

Leap seconds are occasionally manually scheduled. Currently, computer clocks have no facility to honour leap seconds; there is no standard to tell them up-front to insert one. Instead, computer clocks periodically re-synch their time keeping via the NTP protocol and adjust automatically after the leap second has been inserted.

Next, computer clocks usually report the time as seconds since the epoch. It'd be up to the datetime module to adjust its accounting when converting that second count to include leap seconds. It doesn't do this at present. time.time() will just report a time count based on the seconds-since-the-epoch.

So, nothing different will happen when the leap second is officially in effect, other than that your computer clock will be off by 1 second for a little while.

The issues with datetime only cover representing a leap second timestamp, which it can't. It won't be asked to do so anyway.

Paul Gear
  • 236
  • 4
  • 12
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 1
    So does this mean that it just "not specified" and your code needs to be aware of how the NTP will be placed for your machine? (Assuming you want to keep using `datetime` and not move to a library like `astropy.time`, thanks @MaxNoe) – Mario Corchero Sep 25 '16 at 12:19
  • 2
    @MarioCorchero: no, I didn't say it is not specified. Your computer clock doesn't know about leap seconds, that whole issue is simply ignored. Your computer clock is not nearly accurate enough *anyway*, which is why NTP is used. Your computer adjusts its clock periodically via that protocol all the time. The only thing you need to do in code is not assume that time always goes forward (as NTP adjustments and time zones can cause time to step backwards). – Martijn Pieters Sep 25 '16 at 12:21
  • @MarioCorchero: for the vast majority of programs, this is entirely a non-issue. If you have a use-case that requires *absolute timing precisions* where UTC timestamps are of crucial importance, don't use Python and don't rely on the computer clock. Get yourself set up with specialised timing equipment instead. – Martijn Pieters Sep 25 '16 at 12:22
  • Thanks, on my specific use case this behaviour is fine as well, I just wanted to understand how it will behave as I have some to do some arithmetics with time. – Mario Corchero Sep 25 '16 at 12:25
  • 1
    I think, saying that one’s computer does not know about leap seconds is a bit of an oversimplification. It is actually the opposite: computer clocks _do_ have a facility for “honouring” leap seconds (in their own weird way): NTP. If computers did not know about leap seconds, UNIX time would equal the number of seconds passed in the real world since the epoch, but because computers do account for leap seconds (in a terrible way), the UNIX time is currently _less_ than what you would see if you started a stopwatch on the epoch in the real world and looked at it now. – kirelagin Mar 18 '21 at 19:39
  • @kirelagin: NTP just calibrates the computer time. You can't, from software running on the computer, 'know' if a leap second was applied and so there is no API for this. The Python `datetime` module just deals with UNIX timestamps. – Martijn Pieters Mar 21 '21 at 12:27
3

Python's mktime behavior here is pretty much just inherited from C's, which is specified by POSIX. The spec isn't as simple as you might think, and the reality is even more complicated.

I think this article is a good introduction to the various issues: The Unix leap second mess

The datetime module, as you observed, just acts like there is no such thing as a leap second.

wjandrea
  • 28,235
  • 9
  • 60
  • 81
morganwahl
  • 1,668
  • 1
  • 11
  • 5