2

I find the datetime.timestamp() function return different value on Linux and Windows. Here is a simple to to replicate it:

from datetime import date, time, datetime, timedelta

def main(): 
    dt = datetime(2000, 1, 1)
    edt = datetime(2006, 12, 31)

    fname = 't1.csv'
    f = open(fname, 'w')
    f.write('date,timestamp\n')
    while dt <= edt:
        f.write('{0:%Y-%m-%d},{1:.0f}\n'.format(dt, dt.timestamp()))
        dt += timedelta(days=1)
    f.close()

return 0

Here is the LAST different part from Windows: (Windows7 64 + Python3.4.3 64)

...
2006-10-30,1162180800
2006-10-31,1162267200
2006-11-01,1162353600
2006-11-02,1162440000
2006-11-03,1162526400
2006-11-04,1162612800
2006-11-05,1162699200
...

Here is the corresponding Linux output: (RedHat6 64 + Python 3.4.3 64)

...
2006-10-30,1162184400
2006-10-31,1162270800
2006-11-01,1162357200
2006-11-02,1162443600
2006-11-03,1162530000
2006-11-04,1162616400
2006-11-05,1162702800
...

Systems all using the EST with automatic DST adjustment. Some observations:

  • the linux output seems to be correct
  • no difference observed after 2006 (I did not test above 12/31/2015)
  • the difference seems to be 1 hour/3600 seconds
  • the difference seems to happen only in those days around the DST change in that year (or around the time EU/US has different DST change date.)

Just wondering why timestamp() function behaves differently on windows and linux.

fivelements
  • 1,487
  • 2
  • 19
  • 32
  • 1
    In Linux, the time zone updates are managed very carefully, and any quirks introduced by national governments get there very quickly. But not necessarily in the actual distributions if they are out of support. For Linux, what you would need to troubleshoot is the version of the `tzdata` package (`rpm -qa | grep tzdata`). And the time zones are actually city/region based, so check what exactly was it set to; one way is `find /usr/share/zoneinfo -type f|xargs md5sum|grep $(md5sum /etc/localtime|cut -c1-20)|awk '{print $2}'` - may be more than one. But no clue under Windows.. – chexum Jan 05 '16 at 22:38
  • I understand what you mean. I always use 'America/New_York' in our prod system. Just happen to notice this issue when code/testing some old data points on Windows. BTW, for java, its consistent on Linux/windows and result is the same as Linux+Python. I think JDK also maintains its own tzinfo. – fivelements Jan 05 '16 at 23:47
  • related: [Converting datetime.date to UTC timestamp in Python](http://stackoverflow.com/q/8777753/4279) – jfs Jan 06 '16 at 04:47

2 Answers2

4

datetime.timestamp() on a naive datetime object calls mktime() internally i.e., the input is interpreted as the local time. Local time definitions may differ between systems.

C mktime() may return a wrong result if the local timezone had different utc offset in the past and a historical timezone database is not used. python has no access to the tz database on Windows.

You may get different results if applications use different tzdata versions. You may also get different results for ambiguous times (e.g., during DST transitions) if different mktime() implementations are used (all else being equal).

To get the same result on different systems, use pytz module (the same version on different systems that uses bundled with the Python package zoneinfo):

#!/usr/bin/env python3
from datetime import datetime
import pytz  # $ pip install pytz

tz = pytz.timezone('America/New_York')
for tt in [(2006, 10, 30),
           (2006, 10, 31),
           (2006, 11, 1),
           (2006, 11, 2),
           (2006, 11, 3),
           (2006, 11, 4),
           (2006, 11, 5)]:
    dt = datetime(*tt)
    ts = tz.localize(dt, is_dst=None).timestamp()
    print("{dt:%Y-%m-%d},{ts:.0f}".format(**vars()))

Output (pytz.__version__ == 2014.10)

2006-10-30,1162184400
2006-10-31,1162270800
2006-11-01,1162357200
2006-11-02,1162443600
2006-11-03,1162530000
2006-11-04,1162616400
2006-11-05,1162702800
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • Thanks! Using tz.localize() is the correct way to do it cross platform. On the other hand, localize is very slow. On windows7, it is 25 times slower if I use tz.localze().timestamp() vs datetime.timestamp(). Do you know if datetime.timestamp() will be consistent on RedHat/CentOS system for historical days? Based on the date range 2000-2015 it looks good. – fivelements Jan 07 '16 at 15:23
  • This is why we moved from localize to simple timestamp(): http://stackoverflow.com/questions/33532457/pytz-timezone-conversion-performance?noredirect=1#comment54870338_33532457 – fivelements Jan 07 '16 at 15:30
  • For us (and most people), the datetime we get/use is based on ONE timezone which is the default system timezone. Because each language has different datetime representation, what we really need is a fast way to standardize and pass the datetime around. EPOCH seconds/milliseconds seems to be the choice. Using the result from date.timestamp(), I can easily construct a java date: new Date(milli), without any loss of info. The underlying assumption is all these systems live in the SAME timezone. (not necessary UTC timezone.) – fivelements Jan 07 '16 at 15:40
  • @fivelements: yes, most software is broken. If you need to optimize for speed then it is a different question (you can find a less general fast solution that is still correct in most cases) – jfs Jan 07 '16 at 15:50
0

From doing some research on your issue I found this link it talks about how the start dates for systems are different I would think that linux is like unix and uses 01/01/1970 as its start time and windows uses a time much earlier and dependent on what you are doing as from what I gather windows is either 01/01/1601 if you are saving files to the file system but the time used to calculate a timestamp is 31/12/1601. So it looks like a difference in what they see as the start of time if you take a further look you will see that the actual start times are different in terms of 12:00:00AM for example.

bobthemac
  • 1,172
  • 6
  • 26
  • 59
  • difference is only an hour, not 369 years – cmd Jan 05 '16 at 22:58
  • but if you look the actual start time is different by about an hour – bobthemac Jan 05 '16 at 23:07
  • I think prob it relates to how python loads the system tzinfo db, and how each system maintains the tzinfo – fivelements Jan 05 '16 at 23:42
  • `datetime.timestamp()` returns "seconds since epoch" on both Linux and Windows. FILETIME is unrelated here (python converts from 1601 timestamp to 1970 timestamp automatically). – jfs Jan 06 '16 at 04:35
  • Thanks for that didn't know that don't know what the link I posted is talking about he converts the dates – bobthemac Jan 06 '16 at 10:07