normalizing JSON datestrings to UTC python

Question

I have an important test that says "Calculate users that logged in during the month of April normalized to the UTC timezone."

Items look as such:

[ {u'email': u' ybartoletti@littel.biz',
  u'login_date': u'2014-05-08T22:30:57-04:00'},
 {u'email': u'woodie.crooks@kozey.com',
  u'login_date': u'2014-04-25T13:27:48-08:00'},
]

It seems to me that an item like 2014-04-13T17:12:20-04:00 means "April 13th, 2014, at 5:12:20 pm, 4 hours behind UTC". Then I just use strptime to convert to datetime (Converting JSON date string to python datetime), and subtract a timedelta of however many hours I get from a regex that grabs the end of string? I feel this way because some have a + at the end instead of -, like 2014-05-07T00:30:06+07:00

Thank you

score 3 · Accepted Answer · answered Mar 06 '16 at 00:17

3

It is probably best to use the dateutil.parser.parse and pytz packages for this purpose. This will allow you to parse a string and convert it to a datetime object with UTC timezone:

>>> s = '2014-05-08T22:30:57-04:00'
>>> import dateutil.parser
>>> import pytz
>>> pytz.UTC.normalize(dateutil.parser.parse(s))
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=<UTC>)

answered Mar 06 '16 at 00:17

Simeon Visser

118,920
18
185
180

Is there a major difference between `pytz.UTC.normalize(t)` and `t.astimezone(pytz.UTZ)` as I had in my (now deleted) answer? – mgilson Mar 06 '16 at 00:19
@mgilson: in my understanding not for UTC but it could make a difference for other timezones. In the `pytz` documentation I also see the following being used: `au_dt = au_tz.normalize(utc_dt.astimezone(au_tz))` - I'm not sure why it is done that way. – Simeon Visser Mar 06 '16 at 00:24
wow, as you were answering I just wasted 15 mins and 10 lines of code lol. very nice, let me test – codyc4321 Mar 06 '16 at 00:38
that was so beautiful. I haven't seen that parser before simeon – codyc4321 Mar 06 '16 at 00:44
This bug report also seems to be related: https://answers.launchpad.net/pytz/+question/249229 – mgilson Mar 06 '16 at 03:48
Hmmm ... I was under the same impression as you, but (unless I'm reading it incorrectly) the source seems to imply that astimezone should be used for conversions (even though normalize also works)... I'm still trying to wrap my head around this one though. I have timezones ... – mgilson Mar 06 '16 at 04:46

uhriab · Answer 2 · 2016-03-06T11:05:56.137

2

You can use arrow to easily parse date with time zone.

>>>import arrow
>>> a = arrow.get('2014-05-08T22:30:57-04:00').to('utc')
>>> a
<Arrow [2014-05-09T02:30:57+00:00]>

Get a datetime object or timestamp:

>>> a.datetime
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=tzutc())
>>> a.naive
datetime.datetime(2014, 5, 9, 2, 30, 57)
>>> a.timestamp
1399602657

edited Mar 06 '16 at 11:05

answered Mar 06 '16 at 00:24

uhriab

21
4

1

is that datetype specific to arrow? does arrow have a convert func to python? – codyc4321 Mar 06 '16 at 00:44
@codyc4321 Arrow datetype specific for module, but you can easily get datetime object if you need – uhriab Mar 06 '16 at 11:00

score 1 · Answer 3 · answered Mar 06 '16 at 00:57

The following solution should be faster and avoids importing external libraries. The downside is that it will only work if the date strings are all guaranteed to have the specified format. If that's not the case, then I would prefer Simeon's solution, which lets dateutil.parser.parse() take care of any inconsistencies.

import datetime as dt

def parse_date(datestr):
    diff = dt.timedelta(hours=int(datestr[20:22]), minutes=int(datestr[23:]))
    if datestr[19] == '-':
        return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') - diff
    return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') + diff

it seems faster and more native but it's not guaranteed in this case, I didn't write the api — codyc4321, Mar 06 '16 at 02:22

normalizing JSON datestrings to UTC python

3 Answers3