App engine datastore to_dict alternative to serialize ndb.Model to JSON

Question

I have a ndb.Model that I want to convert to JSON.

class Users(ndb.Model):
    username = ndb.StringProperty(indexed=True)
    password= ndb.StringProperty(indexed=True)
    created_at = ndb.DateTimeProperty(auto_now_add=True)

user = Users.query(Users.username==username).get()
rv = json.dumps(user.to_dict()) 
print(rv)

It throws this error:

 TypeError: datetime.datetime(2013, 11, 24, 3, 40, 15) is not JSON serializable

Most of the solutions here are for db.Model and are fairly outdated.

sdk version 1.9.10

Unrelated comment: you can use `.get()` instead of `.fetch(1)` if you only need one result, that way you don't end up with a list of one element, i.e. later you can use `user.to_dict()` instead of `user[0].to_dict()`. — Mihail Russu, Sep 16 '14 at 14:47

Max Tsepkov · Answer 1 · 2019-02-13T14:44:02.163

You can extend Property classes to handle special cases. It is applicable for any property type.

from google.appengine.ext import ndb

class DateTimeProperty(ndb.DateTimeProperty):
    # Override to allow JSON serialization
    def _get_for_dict(self, entity):
        value = super(DateTimeProperty, self)._get_for_dict(entity);
        return value.isoformat()

Then use it in your model:

class Users(ndb.Model):
    username = ndb.StringProperty(indexed=True)
    password= ndb.StringProperty(indexed=True)
    created_at = DateTimeProperty(auto_now_add=True)

And to_dict() as you would normally do:

user = Users.query(Users.username==username).get()
user.to_dict()

This is definetly the neatest solution. Thanks! – tuned May 13 '16 at 13:52 — tuned, May 13 '16 at 13:52

Mihail Russu · Accepted Answer · 2014-09-16T15:12:25.687

You would need a custom "to JSON" converter that handles formats not natively supported by JSON.

I am using something like the following code that handles most situations for me.

def to_json(self, o):
    if isinstance(o, list):
        return [self.to_json(l) for l in o]
    if isinstance(o, dict):
        x = {}
        for l in o:
            x[l] = self.to_json(o[l])
        return x
    if isinstance(o, datetime.datetime):
        return o.isoformat()
    if isinstance(o, ndb.GeoPt):
        return {'lat': o.lat, 'lon': o.lon}
    if isinstance(o, ndb.Key):
        return o.urlsafe()
    if isinstance(o, ndb.Model):
        dct = o.to_dict()
        dct['id'] = o.key.id()
        return self.to_json(dct)
    return o

So in my case I am also taking care of some other things like GeoPt, and adding an ID field to all ndb.Models but for your case all you'd need would be:

    if isinstance(o, datetime.datetime):
        return o.isoformat()

but I am guessing (not really sure) you would then get a key error as well so you'd also need

    if isinstance(o, ndb.Key):
        return o.urlsafe()

In case if you didn't need the created_at field, you could simply exclude it like

rv = json.dumps(user.to_dict(exclude=['created_at']))

still raise an error : TypeError: datetime.datetime(2013, 11, 24, 3, 40, 15) is not JSON serializable, i use python2.7 and gae sdk for linux 1.9.10 — jvaldenor, Sep 16 '14 at 15:03
How are you using it? Instead of `json.dumps(user.to_dict())` you would need to have something like `json.dumps(self.to_json(user))`. — Mihail Russu, Sep 16 '14 at 15:05
json.dumps has an argument, 'default', that can take a function to be called for otherwise unserializable types, letting you keep your choice of which properties to include in the json separate from the serialization. — Greg, Sep 16 '14 at 17:28

score 1 · Answer 3 · edited May 23 '17 at 11:46

Another possible solution - inspired from this answer - is to override to_dict() in your User class:

class User(ndb.Model):
    username = ndb.StringProperty(indexed=True)
    password = ndb.StringProperty(indexed=False)
    created_at = DateTimeProperty(auto_now_add=True)

    def to_dict(self):
        result = super(User, self).to_dict()
        result['created_at'] = self.created_at.isoformat()
        return result

App engine datastore to_dict alternative to serialize ndb.Model to JSON

3 Answers3