6

In Python, I can find the Unix time stamp of a local time, knowing the time zone, like this (using pytz):

>>> import datetime as DT
>>> import pytz
>>> mtl = pytz.timezone('America/Montreal')
>>> naive_time3 = DT.datetime.strptime('2013/11/03', '%Y/%m/%d')
>>> naive_time3
datetime.datetime(2013, 11, 3, 0, 0)
>>> localized_time3 = mtl.localize(naive_time3)
>>> localized_time3
datetime.datetime(2013, 11, 3, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>)
>>> localized_time3.timestamp()
1383451200.0

So far, so good. naive_time is not aware of the time zone, whereas localized_time knows its midnight on 2013/11/03 in Montréal, so the (UTC) Unix time stamp is good. This time zone is also my local time zone and this time stamp seems right:

$ date -d @1383451200
Sun Nov  3 00:00:00 EDT 2013

Now, clocks were adjusted one hour backward November 3rd at 2:00 here in Montréal, so we gained an extra hour that day. This means that there were, here, 25 hours between 2013/11/03 and 2013/11/04. This shows it:

>>> naive_time4 = DT.datetime.strptime('2013/11/04', '%Y/%m/%d')
>>> localized_time4 = mtl.localize(naive_time4)
>>> localized_time4
datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EST-1 day, 19:00:00 STD>)
>>> (localized_time4.timestamp() - localized_time3.timestamp()) / 3600
25.0

Now, I'm looking for an easy way to get the localized_time4 object from localized_time3, knowing I want to get the next localized day at the same hour (here, midnight). I tried timedelta, but I believe it's not aware of time zones or DST:

>>> localized_time4td = localized_time3 + DT.timedelta(1)
>>> localized_time4td
datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>)
>>> (localized_time4td.timestamp() - localized_time3.timestamp()) / 3600
24.0

My purpose is to get informations about log entries that are stored with their Unix timestamp for each local day. Of course, if I use localized_time3.timestamp() and add 24 * 3600 here (which will be the same as localized_time4td.timestamp()), I will miss all log entries that happened between localized_time4td.timestamp() and localized_time4td.timestamp() + 3600.

In other words, the function or method I'm looking for should know when to add 25 hours, 24 hours or 23 hours sometimes to a Unix time stamp, depending on when DST shifts happen.

eepp
  • 7,255
  • 1
  • 38
  • 56
  • Does it have to be std lib modules or can you use pip modules? – rdodev Nov 29 '13 at 00:31
  • I'm willing to use lightweight PyPI modules if they provide a really easy solution. For instance, I'm planning on using `pytz`. – eepp Nov 29 '13 at 00:37
  • 1
    May I recommend you take a look at the excellent Arrow (http://crsmithdev.com/arrow/). – rdodev Nov 29 '13 at 00:38
  • @rdodev: Arrow seems to work great (just tried it)! Do you want to provide an answer (that I will accept) with a working example, or should I answer my own question using Arrow? Thank you anyway. – eepp Nov 29 '13 at 01:00
  • glad to help! Go ahead and answer it yourself. I'm a bit short on time. – rdodev Nov 29 '13 at 01:02
  • Realize that if you start with a time within the DST changeover hour you may get an impossible result because the time does not exist, or an indeterminate one because the time exists twice. – Mark Ransom Dec 02 '13 at 17:38

4 Answers4

3

Without using a new package:

def add_day(x):
    d = x.date()+DT.timedelta(1)
    return mtl.localize(x.replace(year=d.year, month=d.month, day=d.day, tzinfo=None))

Full script:

import datetime as DT
import pytz
import calendar
mtl = pytz.timezone('America/Montreal')
naive_time3 = DT.datetime.strptime('2013/11/03', '%Y/%m/%d')
print repr(naive_time3)
#datetime.datetime(2013, 11, 3, 0, 0)
localized_time3 = mtl.localize(naive_time3)
print repr(localized_time3)
#datetime.datetime(2013, 11, 3, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>)
print calendar.timegm(localized_time3.utctimetuple())
#1383451200.0
def add_day(x):
    d = x.date()+DT.timedelta(1)
    return mtl.localize(x.replace(year=d.year, month=d.month, day=d.day, tzinfo=None))
print repr(add_day(localized_time3))
#datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EST-1 day, 19:00:00 STD>)

(calendar is for Python2.)

cyborg
  • 9,989
  • 4
  • 38
  • 56
  • I actually like this solution! It's not as trivial as with using Arrow, which really has this feature built-in, but it's still doing the work without external deps. Thanks! – eepp Nov 29 '13 at 16:44
  • it doesn't handle ambiguous or non-existent time. You could [fix it using `.dst(), `.normalize()` methods](http://stackoverflow.com/a/20333758/4279) – jfs Dec 02 '13 at 17:04
  • Thanks for pointing this out, J.F. Sebastian. However, in my specific case, I only need to support midnight (i.e. the user only specifies a date, not a date/time). Therefore, ambiguous/non-existent times will never be a problem. It's interesting to know there's also a solution for this in case one needs it, given in your answer to my question, though. – eepp Dec 02 '13 at 18:02
  • @eepp: Timezone utc offset may change at midnight (it is not common unless you are in Brasilia but [it happens](http://stackoverflow.com/a/6841479/4279)). If you want to assert that your code doesn't handle it then use `is_dst=None` to detect it. To get midnight: `datetime.combine(some_date, time(0, 0))` could be used instead of `x.replace(year=d.year, ...`. – jfs Dec 02 '13 at 18:55
2

I gradually provide several solutions with the most robust solution at the very end of this answer that tries to handle the following issues:

  • utc offset due to DST
  • past dates when the local timezone might have had different utc offset due to reason unrelated to DST. dateutil and stdlib solutions fail here on some systems, notably Windows
  • ambiguous times during DST (don't know whether Arrow provides interface to handle it)
  • non-existent times during DST (the same)

To find POSIX timestamp for tomorrow's midnight (or other fixed hour) in a given timezone, you could use code from How do I get the UTC time of “midnight” for a given timezone?:

from datetime import datetime, time, timedelta
import pytz

DAY = timedelta(1)
tz = pytz.timezone('America/Montreal')

tomorrow = datetime(2013, 11, 3).date() + DAY

midnight = tz.localize(datetime.combine(tomorrow, time(0, 0)), is_dst=None)
timestamp = (midnight - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

dt.date() method returns the same naive date for both naive and timezone-aware dt objects.

The explicit formula for timestamp is used to support Python version before Python 3.3. Otherwise .timestamp() method could be used in Python 3.3+.

To avoid ambiguity in parsing input dates during DST transitions that are unavoidable for .localize() method unless you know is_dst parameter, you could use Unix timestamps stored with the dates:

from datetime import datetime, time, timedelta
import pytz

DAY = timedelta(1)
tz = pytz.timezone('America/Montreal')

local_dt = datetime.fromtimestamp(timestamp_from_the_log, tz)
tomorrow = local_dt.date() + DAY

midnight = tz.localize(datetime.combine(tomorrow, time(0, 0)), is_dst=None)
timestamp = (midnight - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

To support other fixed hours (not only midnight):

tomorrow = local_dt.replace(tzinfo=None) + DAY # tomorrow, same time
dt_plus_day = tz.localize(tomorrow, is_dst=None)
timestamp = dt_plus_day.timestamp() # use the explicit formula before Python 3.3

is_dst=None raises an exception if the result date is ambiguous or non-existent. To avoid exception, you could choose the time that is closest to the previous date from yesterday (same DST state i.e., is_dst=local_dt.dst()):

from datetime import datetime, time, timedelta
import pytz

DAY = timedelta(1)
tz = pytz.timezone('America/Montreal')

local_dt = datetime.fromtimestamp(timestamp_from_the_log, tz)
tomorrow = local_dt.replace(tzinfo=None) + DAY

dt_plus_day = tz.localize(tomorrow, is_dst=local_dt.dst())
dt_plus_day = tz.normalize(dt_plus_day) # to detect non-existent times                                            
timestamp = (dt_plus_day - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

.localize() respects given time even if it is non-existent, therefore .normalize() is required to fix the time. You could raise an exception here if normalize() method changes its input (non-existent time detected in this case) for consistency with other code examples.

Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • Why would you do `is_dst=local_dt.dst()` to ensure the "time that is closest to the previous date from yesterday"? The `is_dst` parameter of `localize` accepts `True`, `False` or `None`, while the `dst()` method of a `datetime.datetime` returns a `datetime.timedelta`. Thus, `mtl.localize(DT.datetime(2013, 11, 4)).dst() == True` is `False`, and so is `mtl.localize(DT.datetime(2013, 11, 4)).dst() == True`, whereas the first has DST and the second does not. – eepp Dec 03 '13 at 23:00
  • 1
    @eepp: Avoid `some_python == True`, try `bool(some_python_obj)` instead. Read [Truth value testing in Python](http://docs.python.org/2/library/stdtypes.html#truth-value-testing) – jfs Dec 03 '13 at 23:09
  • Good to know! I'm fairly new to Python, but getting better thanks to guys like you. Here's what the doc says: *and in Boolean contexts, a `timedelta` object is considered to be true if and only if it isn’t equal to `timedelta(0).`*. This makes sense now. You really shed light on something however, thank you. – eepp Dec 03 '13 at 23:47
  • @eepp: just to be clear: you rarely need to call `bool()` explicitly, `if some_python_obj: ...` works as is. – jfs Dec 04 '13 at 00:16
1

(Thanks to @rdodev for pointing me to Arrow).

Using Arrow, this operation becomes easy:

>>> import arrow
>>> import datetime as DT
>>> lt3 = arrow.get(DT.datetime(2013, 11, 3), 'America/Montreal')
>>> lt3
<Arrow [2013-11-03T00:00:00-04:00]>
>>> lt4 = arrow.get(DT.datetime(2013, 11, 4), 'America/Montreal')
>>> lt4
<Arrow [2013-11-04T00:00:00-05:00]>
>>> lt4.timestamp - (lt3.replace(days=1).timestamp)
0
>>> (lt3.replace(days=1).timestamp - lt3.timestamp) / 3600
25.0

Using Arrow's replace method, singular unit names replace that property while plural adds to it. So lt3.replace(days=1) is November 4th, 2013 while lt3.replace(day=1) is November 1st, 2013.

eepp
  • 7,255
  • 1
  • 38
  • 56
  • How does `Arrow` deal with [ambiguous or non-existent times](http://pytz.sourceforge.net/#problems-with-localtime)? – jfs Dec 02 '13 at 17:13
  • Well, November 3rd, 2013 at 1:30 in Montréal seems to default to no DST because it outputs `` (while 0:59 has DST: ``). – eepp Dec 02 '13 at 18:14
  • what about non-existent time such as `2014-03-09 02:45:00`? – jfs Dec 02 '13 at 19:19
1

Here an alternative based on dateutil:

>>> # In Spain we changed DST 10/26/2013
>>> import datetime
>>> import dateutil.tz
>>> # tzlocal gets the timezone of the computer
>>> dt1 = datetime.datetime(2013, 10, 26, 14, 00).replace(tzinfo=dateutil.tz.tzlocal())
>>> print dt1
2013-10-26 14:00:00+02:00
>>> dt2 = dt1 + datetime.timedelta(1)
>>> print dt2
2013-10-27 14:00:00+01:00
# see if we hace 25 hours of difference
>>> import time
>>> (time.mktime(dt2.timetuple()) - time.mktime(dt1.timetuple())) / 3600.0
25.0
>>> (float(dt2.strftime('%s')) - float(dt1.strftime('%s'))) / 3600   # the same
25.0
erny
  • 1,296
  • 1
  • 13
  • 28
  • [`dateutil` might fail during DST transitions](https://gist.github.com/zed/3838828). It might fail for past dates if OS' historical timezone database is not used or absent (Windows). It doesn't handle ambiguous, non-existent times. – jfs Dec 02 '13 at 17:13
  • I tested DST transitions with my timezone (CEST -> CET) as the example demonstrates. I'm using it for recent and future dates only. Thanks for the tip. – erny Dec 02 '13 at 23:06
  • timezone info changes all the time. There are [recent changes in 2013](https://pypi.python.org/pypi/pytz/). I'm sure the changes won't stop in the future. – jfs Dec 02 '13 at 23:44