Python JSON encoder to support datetime?

Question

is there any elegant way to make Python JSON encoder support datetime? some 3rd party module or easy hack?

I am using tornado's database wrapper to fetch some rows from db to generate a json. The query result includes a regular MySQL timestamp column.

It's quite annoying that Python's default json encoder doesn't support its own datetime type, which is so common in all kinds of database queries.

I don't want to modify Python's own json encoder. any good practice? Thanks a lot!

ps: I found a dirty hack by modifying the Python JSON encoder default method:

Change:

def default(self, o):
    raise TypeError(repr(o) + " is not JSON serializable")

To:

def default(self, o):
    from datetime import date
    from datetime import datetime
    if isinstance(o, datetime):
        return o.isoformat()
    elif isinstance(o, date):
        return o.isoformat()
    else:
        raise TypeError(repr(o) + " is not JSON serializable")

well, it will be a temporary solution just for dev environment.

But for long term solution or production environment, this is quite ugly, and I have to do the modification every time I deploy to a new server.

Is there a better way? I do not want to modify Python code itself, neither Tornado source code. Is there something I can do with my own project code to make this happen? preferably in one pace.

Thanks a lot!

See: http://stackoverflow.com/questions/455580/json-datetime-between-python-and-javascript — Peter Hoffmann, Aug 25 '12 at 12:49
the problem with the subclass method, is that it fails for all the other uses of json encoding, such as simple django, "dumpdata" — mcr, Nov 21 '12 at 18:33

score 65 · Answer 1 · answered Oct 04 '14 at 17:38

65

json.dumps(thing, default=str)

answered Oct 04 '14 at 17:38

David Jones

4,766
3
32
45

6

please upvote if you think this meets the definition of "easy hack" mentioned in the question. – David Jones Nov 04 '14 at 14:32
3

I guess this should be the correct answer, Wonder what is wrong with this answer? – titogeo Oct 09 '18 at 11:04
4

I find it amusing. 5 years later and this is still the most simple and Pythonic way to overcome the issue, and this answer is still underrated. – Franco Roura May 03 '19 at 03:37
this actually preserves the float and int values when `thing` is a `Dict`. – nurettin Nov 13 '19 at 13:04
1

you would rather agree on some common formatting and use `strftime` inside some function you're passing into `default` – Sławomir Lenart Feb 15 '21 at 13:26

Cole Maclean · Accepted Answer · 2023-01-03T23:33:53.090

32

The docs suggest subclassing JSONEncoder and implementing your own default method. Seems like you're basically there.

The reason dates aren't handled by the default encoder is there is no standard representation of a date in JSON. Some people are using the format /Date(1198908717056)/, but I prefer ISO format personally.

import json
import datetime


class DateTimeEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, (datetime.datetime, datetime.date, datetime.time)):
            return obj.isoformat()
        elif isinstance(obj, datetime.timedelta):
            return (datetime.datetime.min + obj).time().isoformat()

        return super(DateTimeEncoder, self).default(obj)

now = datetime.datetime.now()
encoder = DateTimeEncoder()
encoder.encode({"datetime": now, "date": now.date(), "time": now.time()})
> {"datetime": "2019-07-02T16:17:09.990126", "date": "2019-07-02", "time": "16:17:09.990126"}

edited Jan 03 '23 at 23:33

answered Aug 26 '12 at 01:21

Cole Maclean

5,627
25
37

1

You can combine the `if` and first `elif` by taking advantage of the fact that you do the same thing for both `return obj.isoformat()` and that `isinstance` supports tuples: `isinstance(obj, (datetime.datetime, datetime.date))` – Bailey Parker Apr 12 '19 at 05:21
this worked for me, I added the additional checks into my existing custom JSON encoder – Raveen Beemsingh Aug 01 '19 at 11:41
This `timedelta` encoding fails for values over (or equal to) 24 hours. E.g. for a `timedelta(hours=24)`, the encoded value will be `00:00:00` (0 hours), or for `timedelta(hours=25)`, it will be `01:00:00` (1 hour). – cod3monk3y Dec 11 '19 at 06:10
@cod3monk3y if you have timedeltas over 24h, use the `total_seconds` method, or do manual string formatting. isoformat is an easy trick for stort ones. – Cole Maclean Dec 15 '19 at 20:29

SzieberthAdam · Answer 3 · 2014-04-29T09:59:19.350

I made my own classes for my project:

import datetime
import decimal
import json
import sys

class EnhancedJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            ARGS = ('year', 'month', 'day', 'hour', 'minute',
                     'second', 'microsecond')
            return {'__type__': 'datetime.datetime',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.date):
            ARGS = ('year', 'month', 'day')
            return {'__type__': 'datetime.date',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.time):
            ARGS = ('hour', 'minute', 'second', 'microsecond')
            return {'__type__': 'datetime.time',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.timedelta):
            ARGS = ('days', 'seconds', 'microseconds')
            return {'__type__': 'datetime.timedelta',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, decimal.Decimal):
            return {'__type__': 'decimal.Decimal',
                    'args': [str(obj),]}
        else:
            return super().default(obj)


class EnhancedJSONDecoder(json.JSONDecoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, object_hook=self.object_hook,
                         **kwargs)

    def object_hook(self, d): 
        if '__type__' not in d:
            return d
        o = sys.modules[__name__]
        for e in d['__type__'].split('.'):
            o = getattr(o, e)
        args, kwargs = d.get('args', ()), d.get('kwargs', {})
        return o(*args, **kwargs)

if __name__ == '__main__':
    j1 = json.dumps({'now': datetime.datetime.now(),
        'val': decimal.Decimal('9.3456789098765434987654567')},
        cls=EnhancedJSONEncoder)
    print(j1)
    o1 = json.loads(j1, cls=EnhancedJSONDecoder)
    print(o1)

Result:

{"val": {"args": ["9.3456789098765434987654567"], "__type__": "decimal.Decimal"}, "now": {"args": [2014, 4, 29, 11, 44, 57, 971600], "__type__": "datetime.datetime"}}
{'val': Decimal('9.3456789098765434987654567'), 'now': datetime.datetime(2014, 4, 29, 11, 44, 57, 971600)}

References:

Note: It can be made more flexible by passing a custom dictionary with types as keys and args, kwargs as values to the encoder's __init__() and use that (or a default dictionary) in the default() method.

Best answer so far. It helps when I was trying to wrap my head around how to deserialize UUIDs. Great! — Daniel Butler, Mar 05 '20 at 21:24
I am glad you are happy with it. I used it couple of times myself. — SzieberthAdam, Mar 05 '20 at 22:35

score 9 · Answer 4 · answered Oct 14 '16 at 14:05

9

json.dumps(r, default=lambda o: o.isoformat() if hasattr(o, 'isoformat') else o)

answered Oct 14 '16 at 14:05

Daniel Lerch

661
7
11

1

While this code snippet may solve the question, [including an explanation](http://meta.stackexchange.com/questions/114762/explaining-entirely-code-based-answers) really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion. – andreas Oct 14 '16 at 19:51

score 2 · Answer 5 · answered May 27 '19 at 12:17

Create a custom decoder/encoder:

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return http_date(obj)
        if isinstance(obj, uuid.UUID):
            return str(obj)
        return json.JSONEncoder.default(self, obj)

class CustomJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        json.JSONDecoder.__init__(self, object_hook=self.object_hook, *args, **kwargs)

    def object_hook(self, source):
        for k, v in source.items():
            if isinstance(v, str):
                try:
                    source[k] = datetime.datetime.strptime(str(v), '%a, %d %b %Y %H:%M:%S %Z')
                except:
                    pass
        return source

score 1 · Answer 6 · answered Aug 25 '12 at 13:02

The Tryton project has a JSONEncoder implementation for datetime.datetime, datetime.date and datetime.time objects (with others). It is used for JSON RPC communication between the server and client.

See http://hg.tryton.org/2.4/trytond/file/ade5432ac476/trytond/protocols/jsonrpc.py#l53

score 0 · Answer 7 · answered Aug 25 '12 at 12:36

0

Convert the datetime type into a unix timestamp, then encode the contents into a json.

e.g. : http://codepad.org/k3qF09Kr

answered Aug 25 '12 at 12:36

DhruvPathak

42,059
16
116
175

you mean changing Python's default datetime type? how to do so? will it be too risky? will it break things? – horacex Aug 25 '12 at 12:37
@horacex no,just modify the resultset that came from tornado database wrapper. – DhruvPathak Aug 25 '12 at 12:40

score -1 · Answer 8 · answered Feb 07 '20 at 14:12

-1

I recommend to use the ujson package or the orjson one.

They are much faster and still support several complex types.

answered Feb 07 '20 at 14:12

AdrianoHRL

26
7

`ujson` didn't work for me (at least not out of the box). I got: `TypeError: datetime.datetime(2020, 4, 18, 17, 50, 42, 418419, tzinfo=datetime.timezone(datetime.timedelta(seconds=10800))) is not JSON serializable` – mcouthon Apr 25 '20 at 09:29
`ujson` doesn't natively serialize `datetime.datetime`. However, `orjson` did the trick. `orjson` also serializes other common types, such as `uuid.UUID`. – RonquilloAeon Nov 19 '20 at 23:01

score -2 · Answer 9 · answered Dec 28 '15 at 07:23

Just create a custom encoder

(the small but important addition to Cole's answer is the handling of pd.NaT (or null/empty timestamp values), since without the addition you will get very weird timestamp conversions for NaT/missing timestamp data)

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if pd.isnull(obj):
            return None
        elif isinstance(obj, datetime):
            return obj.isoformat()
        elif isinstance(obj, date):
            return obj.isoformat()
        elif isinstance(obj, timedelta):
            return (datetime.min + obj).time().isoformat()
        else:
            return super(CustomEncoder, self).default(obj)

Then use it to encode a dataframe:

df_as_dict = df.to_dict(outtype = 'records')  # transform to dict

df_as_json = CustomEncoder().encode(df_as_dict) #transform to json

Since the encoder standardized the data, the regular decoder will act fine in transforming it back to a dataframe:

result_as_dict = json.JSONDecoder().decode(df_as_json) # decode back to dict

result_df = pd.DataFrame(result)  # transform dict back to dataframe

Of course this will also work if you put the dataframe into a larger dict before encoding, e.g

input_dict = {'key_1':val_1,'key_2':val_2,...,'df_as_dict':df_as_dict}
input_json = CustomEncoder().encode(input_dict)
input_json_back_as_dict = json.JSONDecoder().decode(input_json)
input_df_back_as_dict = input_json_back_as_dict['df_as_dict']
input_df_back_as_df = pd.DataFrame(input_df_back_as_dict)

Python JSON encoder to support datetime?

9 Answers9

Linked

Related