Code resulting in weird datetime result on Google App Engine

Question

I'm running a python program on Google App Engine that uses the datetime function. It's supposed to always return UTC time, but it seems to intermittently give an incorrect time. I'm not sure if there's an error with my code, or whether this is an issue on Google's side.

To get my local time (GMT +8:00), I run this function:

def SGTOffset(obj=datetime.now()):
    if isinstance(obj, datetime):
        return obj + timedelta(hours=8)
    return SGTOffset(datetime.now())

and in my main program:

today_date = commoncode.SGTOffset().date()
logging.debug('Today date: %s | Today datetime: %s' % (today_date.isoformat(), commoncode.SGTOffset().isoformat()))

In the logs, I get the following:

[25/Nov/2015:09:00:02 -0800] "GET ... etc ...
01:00:03.287     Today date: 2015-11-25 | Today datetime: 2015-11-25T15:38:20.804300

So, Google kindly formats the log datetime to my locale (GMT +8), showing that the code is run at 01:00:03.287 (26th Nov, GMT +8). Also, this is confirmed by the timestamp 25/Nov/2015:09:00:02 -0800 provided. So the code was run at 25/Nov/2015 17:00:02 UTC time.

However, my code is outputting the wrong time. The datetime that is being generated in the code 2015-11-25T15:38:20.804300 has the timezone of GMT-9:30 instead of UTC time. (Because SGToffset() adds 8 hours to datetime)

This is quite catastrophic as I use the local datetime in many areas of my program. This is also happening intermittently only, because yesterday, the same code ran and got this log:

[24/Nov/2015:09:00:00 -0800] "GET ... etc ...
01:00:02.237     Today date: 2015-11-25 | Today datetime: 2015-11-25T01:00:01.768140

Which is correct! (Google's log timestamp 01:00:02.237 matches the time generated by SGTOffset() which is 01:00:01)

Could I know what is wrong with my program, or whether this is an issue with Google App Engine?

Thank you spending time to read this question!

`datetime.now()` doesn't always return UTC, it returns the time in the servers (local) timezone. `datetime.utcnow()` returns the time in UTC. — mata, Nov 26 '15 at 11:53
According to the [documentation](https://cloud.google.com/appengine/docs/python/datastore/typesandpropertyclasses?csw=1#datetime), `App Engine sets this environment variable to "UTC"`. So, `datetime.now()` should return UTC time. — pipng13579, Nov 26 '15 at 12:25
See, you need to check the documentation to see what the return value is, and then that is only the case for this concrete environment. If you run or test your code or parts of it on a different system then its behaviour may change. Thererfore, if you want to be explicit and consistent, use `utcnow()` when you need the time in utc, use `now()` when it doesn't really mater wheather ist's utc or not. — mata, Nov 26 '15 at 12:43
The `TZ` environment variable set by App Engine should be consistent throughout the whole program. However, I will still convert the code to `utcnow()` as you have suggested. Please do look at the answer I have posted that tackles the root issue. Thanks for helping out! — pipng13579, Nov 26 '15 at 12:53

score 3 · Accepted Answer · edited May 23 '17 at 12:15

The problem lies with the code.

Python stores a default value of the parameter obj for the function SGTOffset() when it is first defined (when the function object is first instantiated), instead of whenever the function is called as I intuitively expected. So, the datetime value will reflect the start time of the instance in GAE.

In order to get the current time whenever I call SGTOffset() without any parameters, I should instead have used:

def SGTOffset(obj=None): # Instead of obj=datetime.now() which is converted to obj='specific datetime' when instantiated
    if isinstance(obj, datetime):
        return obj + timedelta(hours=8)
    return SGTOffset(datetime.now())

In this case, datetime.now() is called dynamically whenever it is required.

I arrived at this solution after viewing a question about old datetime values being returned.

yep, the most typical cause of bugs in users' Python code: forgetting that a default value is computed **once**, at `def` time, not secretly recomputed at every call, as per e.g http://stackoverflow.com/questions/1132941/least-astonishment-in-python-the-mutable-default-argument . The usual way one stumbles into this bug is by using a mutable object as default value, but one of the answers to that question uses exactly `datetime` as the example!-) — Alex Martelli, Nov 27 '15 at 17:58

score 1 · Answer 2 · answered Nov 26 '15 at 14:14

I'm adding a quick answer to give you suggestions to make your code more readable:

obj is not a good variable name because it is not informative
No need for a recursive call to the function
Better to not use isinstance because is None gives you the needed functionality and your code will not work if some other instance type is given anyway.

Here is my suggestion:

def SGTOffset(dt=None):
    if dt is None:
        dt = datetime.now()
    return dt + timedelta(hours=8)

Or if you prefer brevity:

def SGTOffset(dt=None):
    return (dt or datetime.now()) + timedelta(hours=8)

Yep, much improved code here (esp the first, more readable version:-). — Alex Martelli, Nov 27 '15 at 17:59

Code resulting in weird datetime result on Google App Engine

2 Answers2