GAE Datastore date is not JSON serializable

Question

Im trying to send a the 10 latest post from an datastore in app engine.

This is the database:

class Author(ndb.Model):
    """Sub model for representing an author."""
    identity = ndb.StringProperty(indexed=False)
    email = ndb.StringProperty(indexed=True)

class Message(ndb.Model):
    """A main model for representing an individual Message entry."""
    author = ndb.StructuredProperty(Author)
    content = ndb.StringProperty(indexed=False)
    date = ndb.DateTimeProperty(auto_now_add=True)

And this is the code:

class Query(webapp2.RequestHandler):
    def post(self):
        message_name = self.request.get('db_name',
                                         DEFAULT_KEY)
        message_query = Message.query(ancestor=db_key(
                                      message_name)).order(-Message.date)
        messages = message_query.fetch(10)

        items = [] 

        for message in message_query:
            items.append({'id': message.author.identity, 
                          'email':message.author.email,
                          'content':message.content,
                          'date':message.date})    


        self.response.headers['Content-Type'] = 'application/json'   
        self.response.out.write(json.dumps(items))

And the error-message:

TypeError: datetime.datetime(2015, 3, 21, 15, 43, 58, 248650) is not JSON serializable

Why cant JSON return the date on that format? And how do i re-format it to the correct JSON format?

Br

score 5 · Answer 1 · answered Mar 21 '15 at 16:41

Rather than reinventing the wheel, I would recommend installing and using the ndb_json.py module at https://gist.github.com/erichiggins/8969259 -- it robustly and elegantly takes case of all sorts of potential traps and pitfalls on your behalf. Then, ndb_json.dumps is all you need.

Note, though, a peculiar anomaly in the code you post...:

    message_query = Message.query(ancestor=db_key(
                                  message_name)).order(-Message.date)
    messages = message_query.fetch(10)

    items = [] 

    for message in message_query:
        items.append(...)

This will fetch the newest 10 messages into the messages list, which you then completely ignore!, and then all older messages (from looping on the query) get put into items in unbounded numbers. I somehow doubt this is actually what you want -- maybe you meant the for to be:

    for message in messages:
        items.append(...)

to serialize into the response "just the newest 10 messages" rather than "all messages except the newest 10" which is what your code does.

So, if this is the case, with the module I recommend you'd avoid all the "building of items" part, and just replace

self.response.out.write(json.dumps(items))

with

self.response.write(ndb_json.dumps(messages))

(note also that the out part is a legacy artifact/hack for backwards compatibility with ancient versions of webapp and best not used in new apps -- not directly related to your Q but, just like for what I believe is the bug you may have in serializing "the rest of the query" rather than "the newest 10 messages", hopefully helpful to you!-)

score 1 · Answer 2 · edited May 23 '17 at 11:46

1

JSON knows nothing about the datetime objects. It depends on what you want to to with the time later. I would recommend you to convert it to a unix timestamp, which is just an integer and therefore JSON serializable.

Here is a post where the conversion of datetime objects into integers is described for all python versions: Converting datetime.date to UTC timestamp in Python

edited May 23 '17 at 11:46

Community

1
1

answered Mar 21 '15 at 15:54

Christian Eichelmann

968
8
11

score 1 · Accepted Answer · answered Mar 21 '15 at 16:28

You can extend the JSONEncoder class to handle any object type, like this:

from datetime import datetime

import json

class MyJsonEncoder(json.JSONEncoder):
   def default(self, obj):
      if isinstance(obj, datetime):
         # format however you like/need
         return obj.strftime("%Y-%m-%d")
      # pass any other unknown types to the base class handler, probably
      # to raise a TypeError.   
      return json.JSONEncoder.default(self, obj)


objDict = {"str":"foo", "int":212, "date":datetime.today()}

print json.dumps(objDict, cls=MyJsonEncoder)

{"int": 212, "date": "2015-03-21", "str": "foo"}

GAE Datastore date is not JSON serializable

3 Answers3