71

For some reason which I haven't been able to figure out yet, from the the following code:

>>> from pytz import timezone
>>> timezone('America/Chicago')

I get:

<DstTzInfo 'America/Chicago' LMT-1 day, 18:09:00 STD>

When, I assume, I should get:

<DstTzInfo 'America/Chicago' LMT-1 day, 18:00:00 STD>

...since I don't think that my timezone is 6 hours and 9 minutes away from UTC.

I have looked at the source code for pytz but I will admit that I haven't exactly been able to figure out what is going wrong.

I have passed other values to the timezone() function, and the values it returns appear to be correct. For some reason though, the information relevant to my timezone is not correct.

Finally, my co-worker in the cube next to me has confirmed that the function returns the correct timezone info on his machine.

Does anyone have any idea why my timezone ('America/Chicago') would be off by 9 minutes? I am running version 2015.7 of pytz installed using pip. Thank you!

gollum
  • 2,472
  • 1
  • 18
  • 21
elethan
  • 16,408
  • 8
  • 64
  • 87
  • 6
    You are getting the local mean time http://stackoverflow.com/questions/11473721/weird-timezone-issue-with-pytz `tz= timezone('America/Chicago'); tz.localize(datetime.datetime.now())` – Padraic Cunningham Feb 17 '16 at 17:06
  • @PadraicCunningham that definitely seems like what I am experiencing. Any idea why the same code on the machine of the guy next to me gets a different result though? – elethan Feb 17 '16 at 17:31
  • Are you both using the same OS? – Padraic Cunningham Feb 17 '16 at 18:17
  • @PadraicCunningham Different versions of the same OS, Ubuntu 15.10 (me) vs. Ubuntu 14.04 (him). Also, different versions of pytz it turns out. He is using a 2012 version. I am currently trying to figure out how to downgrade to that version to see if that helps. – elethan Feb 17 '16 at 18:21
  • why is it problem, how are you using it? – Padraic Cunningham Feb 17 '16 at 18:22
  • 1
    I would also advise that your friend update not you to downgrade – Padraic Cunningham Feb 17 '16 at 18:31
  • @PadraicCunningham The application I work on in my day job uses the pytz library to get timezone information, and for some reason it calculates timezones in this way only on my machine (not on the production machines or my colleagues' machines). – elethan Feb 17 '16 at 18:33
  • If you normalise the timezone offset will be correct i.e `tz.localize(datetime.datetime.now()).tzinfo` – Padraic Cunningham Feb 17 '16 at 18:37
  • 1
    @PadraicCunningham: do not use `tz.localize(datetime.now())`; use `datetime.now(tz)` instead. The former may fail during DST transitions. – jfs Feb 17 '16 at 18:38
  • @J.F.Sebastian, what is the difference? It is done that way in the pytz docs. – Padraic Cunningham Feb 17 '16 at 18:41
  • @PadraicCunningham: 1- [pytz docs](http://pytz.sourceforge.net/) do not mention `.now()` at all. 2- if you don't understand; ask a separate question (it is unrelated to the current question). – jfs Feb 17 '16 at 18:43
  • @J.F.Sebastian, they mention a naive datetime and pass one to be localized, since `datetime.datetime.now().tzinfo is None` would be True I would consider now to be naive, if there is a problem using now maybe it should be in the documentation. – Padraic Cunningham Feb 17 '16 at 18:48
  • 1
    @PadraicCunningham: think why `localize()` has `is_dst` parameter. Think why you do not need it for the current time (`.now(tz)`). This discussion is not appropriate for the current question. See [ask] – jfs Feb 17 '16 at 18:51
  • Note that even if you provide a concrete instant, the problem persists if you do it the "wrong" way: `datetime(2020, 2, 6, tzinfo=pytz.timezone('America/Chicago'))` gives `2020-02-06 00:00:00-05:51`, but doing `datetime(2020, 2, 6).astimezone(pytz.timezone('America/Chicago'))` gives the expected/correct result. – Ken Williams Feb 05 '20 at 17:05

4 Answers4

73

Answer based on the answer by Carl Meyer in Google Groups Answer

The reason for this difference, is that this is NOT the right way of converting a timezone agnostic datetime object to a timezone aware object.

The explanation being:

"A pytz timezone class does not represent a single offset from UTC, it represents a geographical area which, over the course of history, has probably gone through several different UTC offsets. The oldest offset for a given zone, representing the offset from before time zones were standardized (in the late 1800s, most places) is usually called "LMT" (Local Mean Time), and it is often offset from UTC by an odd number of minutes."

(quote from the cited answer in Google Groups)

Basically, you should do:

from datetime import datetime
import pytz

my_datetime = datetime(2015, 6, 11, 13, 30)
my_tz = pytz.timezone('America/Chicago')    
good_dt = my_tz.localize(my_datetime)

print(good_dt)

out: 2015-06-11 13:30:00-05:00

learn2day
  • 1,566
  • 1
  • 14
  • 17
  • @Antigluk the code in my answer is correct. I see no reason to downvote it. On the other hand the code in this answer may fail for ambiguous or non-existent local times (it may happen in some time zones e.g., during a DST transition) while the code in my answer works even in those cases. – jfs Feb 02 '19 at 18:57
  • 4
    @jfs yeah, you right, downvote wasn't fair. can't remove it. This answer actually encourages everyone to use the localize() method, which usage is much wider than just now(). Without this note first that comes to mind is to use datetime(..., tzinfo=pytz.timezone('America/Chicago')) which is dangerous and leads to use of Local Mean Time. So, I think it's important to find method localize() when googling such things in first place – Antigluk Feb 03 '19 at 19:26
  • @Antigluk "localize" is mentioned explicitly in my answer with a link to more details. – jfs Feb 04 '19 at 15:49
  • 1
    Here's a demo of the dangerous use of Local Mean Time, that I've seen on several accepted answers on StackOverflow. This parameter seems like it is applying the timezone in the same way as localize. Yikes, I wonder how many bugs out there exist due to this: https://repl.it/@jawinn/time-is-off-by-almost-an-hour-when-applying-timezones – jwinn Nov 06 '19 at 22:29
16

Unless your local timezone has a fixed UTC offset then it is pointless to talk about its specific value without providing a specific date/time.

If you provide the time e.g., the current time then you'll see that pytz produces the expected UTC offset:

>>> from datetime import datetime
>>> import pytz
>>> datetime.now(pytz.timezone('America/Chicago')).strftime('%Z%z')
'CST-0600'

See

If you don't provide a specific date/time then pytz may return an arbitrary utc offset from the set of available utc offsets for the given timezone. The recent pytz versions return utc offsets that correspond to the earliest time (LMT as a rule) but you should not rely on it. You and your friend may use different pytz versions that may explain the difference in results.

Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • I assumed that this solution would work and marked it as correct before I figured out how to downgrade to an my colleagues version of `pytz`. Now I have the exact same version as he does, but from an interactive prompt, we get different results from the call to `pytz.timezone('America/Chicago')` (i.e., I get `18:09:00`, he gets `18:00:00`. Any idea what else could account for this discrepancy if it is not the `pytz` version? Could it be a system level configuration difference? – elethan Feb 18 '16 at 16:34
  • @elethan : the code in the answer works. It must produce `'CST-0600'` otherwise you setup is broken. Please, notice the words: "pointless", "should not rely" in the answer—you must provide the date/time e.g., as shown in the example code in the answer. – jfs Feb 18 '16 at 16:38
  • 2
    The code in your answer does work. My issue is that calling `pytz.timezone('America/Chicago')` always returns `` on my machine, and always returns `` on ever other machine I have tested it on (even with the same version of `pytz`) and my initial question was asking what accounts for the difference. I assumed from your answer that it was caused by a difference in versions, but this seems not to be the case. – elethan Feb 18 '16 at 16:44
  • 1- this value (UTC offset from the tz without a specific time) is meaningless. Do you understand that part? 2- this value depends on the pytz version. It might also depend on other things though my guess is that you've tested using a wrong pytz version. It doesn't matter anyway. You should never ever rely on the shown UTC offset without providing the time (if timezone has a non-fixed UTC offset such as America/Chicago). Iff you provide the time; you should get the same value on different platforms (given the same tz database version). – jfs Feb 18 '16 at 16:56
  • 2
    I understand all of that and I agree with you. I was just curious if you might know what could account for this difference with identical versions of `pytz`. Obviously pytz is not randomly choosing an offset if you don't choose give it a specific time, since the results are consistent (even if the method for choosing may potentially change with version). I am wondering why it is choosing a different offset on different machines. – elethan Feb 18 '16 at 17:16
  • 1- on the recent pytz versions (at least since 2014.3), it always uses the UTC offset that corresponds to the earliest time (as I said in the answer). It implies that the value depends on the tz database version too (if your distribution patches pytz to use a non-bundled tzdata). I haven't seen the code for old versions that use a different algorithm (the latest time? — that default may hide bugs). 2- the most likely reason for different results is that you use different pytz versions. Check `pytz.__version__`, `pytz.OLSON_VERSION`. Check that `pytz.__path__` returns an expected value. – jfs Feb 18 '16 at 18:08
  • `pytz.__version__`, `pytz.OLSON_VERSION`, `pytz.__path__` return the same values on all the machines I have tested (i.e., machines that calculate offset as `-0600` and those that calculate it as `-0609`). Oh well, no need to drag this conversation out in the comments any further. I will do some more digging. Thanks again. – elethan Feb 18 '16 at 18:18
15

Just because my curiosity wasn't exactly satisfied, I did a little more digging into this problem recently.

Initially, it seemed that the difference stemmed from different versions of pytz. However, after downgrading my version of pytz to a version where I had confirmed that I got a different result from that on my machine, I found that this wasn't the root of the issue: even with the same version of pytz my machine seemed to be using a UTC offset based on LMT, while the other machines were using one based off CDT or CST.

Based on my conversation with @J.F.Sebastian, I assumed that the only other likely possibility was a system level difference. I dug into the pytz source code a little bit more, and found that the file where pytz gets at least some of it's timezone information from is in /usr/share/zoneinfo/. So I looked at the file /usr/share/zoneinfo/America/Chicago and although it is a binary file, part of it is readable. Half way through the file there is a list of timezones: LMTCDTCSTESTCWTCPT. As you can see, LMT is the first name in the list, and as @J.F.Sebastian suggested, that seems to be the one that pytz uses in the situation described in my original question.

That is how the list looks in Ubuntu 15.10. However, in earlier versions of Ubuntu (e.g., Trusty and Precise) where I was getting the result -600 instead of -609 result, the same list is CDTCSTESTCWTCPT.

I will admit that this comes from a lot of blind exploring and half understanding, but it seems like this is what accounts for the differences I was seeing across machines. As far as why the zoneinfo files differ across versions, and what these differences mean for Ubuntu, I have no idea, but I thought I would share my findings for those who are similarly curious, and to potentially receive insightful corrections/supplemental information from the community.

FObersteiner
  • 22,500
  • 8
  • 42
  • 72
elethan
  • 16,408
  • 8
  • 64
  • 87
  • 1
    note: the stock `pytz` uses a bundled `zoneinfo` (subdirectory). If you see that `/usr/share/zoneinfo` is used; it means you use a modified `pytz` package (e.g., if you've installed `pytz` using `sudo apt-get python-tz` then `pytz` uses `open_resource()` patched in Debian, to use the system zoneinfo from the tzdata package). It is bad if the stock and the patched versions have the same `__version__`, `pytz.OLSON_VERSION` but you shouldn't be surprised that they produce different results (the patched version is probably lying about OLSON_VERSION i.e., the system may use a newer tzdata version). – jfs Mar 04 '16 at 03:05
0

As you mention there are some differences in the original file into the pytz module: (in my case using the Central time)

xxxx......lib/python2.7/site-packages/pytz/zoneinfo/US/Central

In [66]: start = start.replace(tzinfo=central)

In [67]: start.isoformat()
Out[67]: '2018-02-26T00:00:00-05:51'

if you use the standard file of the OS (I tested in mac, ubuntu and centos)

/usr/share/zoneinfo/US/Central

mv xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central-bak

ln -s /usr/share/zoneinfo/US/Central xxxx...../lib/python2.7/site-packages/pytz/zoneinfo/US/Central

The problem is resolved

In [7]: central = timezone('US/Central')

In [8]: central
Out[8]: <DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>

In [10]: start = start.replace(tzinfo=central)

In [11]: start.isoformat()
Out[11]: '2018-02-27T00:00:00-06:00'
Alejandro Guadarrama
  • 1