1337

I have a basic dict as follows:

sample = {}
sample['title'] = "String"
sample['somedate'] = somedatetimehere

When I try to do jsonify(sample) I get:

TypeError: datetime.datetime(2012, 8, 8, 21, 46, 24, 862000) is not JSON serializable

What can I do such that my dictionary sample can overcome the error above?

Note: Though it may not be relevant, the dictionaries are generated from the retrieval of records out of mongodb where when I print out str(sample['somedate']), the output is 2012-08-08 21:46:24.862000.

Phoenix
  • 3,996
  • 4
  • 29
  • 40
Rolando
  • 58,640
  • 98
  • 266
  • 407
  • 1
    Is this specifically python in general, or possibly django? – jdi Aug 09 '12 at 02:05
  • 3
    It technically is specifically python, I am not using django, but retrieving records out of mongodb. – Rolando Aug 09 '12 at 02:05
  • possible duplicate of [JSON datetime between Python and JavaScript](http://stackoverflow.com/questions/455580/json-datetime-between-python-and-javascript) – jdi Aug 09 '12 at 02:05
  • I am using mongoengine, but if pymongo has better ways of getting around this or overcoming this, please tell. – Rolando Aug 09 '12 at 02:07
  • @jdi That answer you linked to as duplicate appears to makes no sense to me. If you can post an answer here that would make sense in my context, that would be more helpful. – Rolando Aug 09 '12 at 02:08
  • That duplicate I posted is the generic way to handle this where you specify the extra parameter to `JSON` as a handler. I have posted a pymongo specific answer. – jdi Aug 09 '12 at 02:10
  • 6
    The linked question is essentially telling you not to try to serialize the datetime object, but rather to convert it to a string in the common ISO format before serializing. – Thomas Kelley Aug 09 '12 at 02:13
  • For a generic solution that handles more that just `datetime.datetime`, see [**_Making object JSON serializable with regular encoder_**](https://stackoverflow.com/questions/18478287/making-object-json-serializable-with-regular-encoder). – martineau Jun 19 '17 at 12:57
  • cannot import name json_util – thang Feb 01 '18 at 05:09
  • If you have the option to store the datetime as a float (seconds since epoch, as from `time.time()`) that is portable and you can call `datetime.datetime.fromtimestamp` when you need a python `datetime`. – kuzzooroo Mar 17 '18 at 21:48

34 Answers34

1607

My quick & dirty JSON dump that eats dates and everything:

json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)

default is a function applied to objects that aren't serializable.
In this case it's str, so it just converts everything it doesn't know to strings. Which is great for serialization but not so great when deserializing (hence the "quick & dirty") as anything might have been string-ified without warning, e.g. a function or numpy array.

jjmontes
  • 24,679
  • 4
  • 39
  • 51
  • 33
    This is awesome, But unfortunately I did not understand what happened? Can anyone explain this answer? – Kishor Pawar Oct 04 '16 at 05:31
  • 138
    @KishorPawar: `default` is a function applied to objects that aren't serializable. In this case it's `str`, so it just converts everything it doesn't know to strings. Which is great for serialization but not so great when deserializing (hence the "quick & dirty") as anything might have been string-ified without warning, e.g. a function or numpy array. – Mark Oct 19 '16 at 20:54
  • 2
    @Mark awesome. Thanks. Useful when you know type of those non-serializable values like dates. – Kishor Pawar Oct 20 '16 at 00:32
  • 2
    Can someone share how to return back to original dictionary? – Muhammad Haseeb Dec 05 '18 at 07:34
  • repr might have a slight advantage over str, but of course there are no guarantees. – c-urchin Mar 21 '19 at 23:12
  • 3
    @jjmontes, doesn't work for everything, eg `json.dumps({():1,type(None):2},default=str)` raises `TypeError`, can't have type or tuple. – alancalvitti Aug 22 '19 at 18:46
  • @c-urchin, `repr` has same issues as `str` for examples in above comment. – alancalvitti Aug 22 '19 at 18:47
  • 1
    I am sorry after this comment only i found that its syntax mistake and not of `default=str`. Thank you for this – Santhosh Aug 29 '19 at 22:24
  • Had to wrap mine in `print(...)` to get it to pretty print. – Danny Bullis Jan 27 '20 at 05:35
  • This saved a year of my life. I would be cautious to use the indent when sending data over APIs – tver3305 May 13 '20 at 05:20
  • This works :-), but when testing to use this also for serialisable data and writing the resulting string to file using `json.dump()`, I find that my whole data set is enclosed in double quotes, and the double quotes around the separate elements are (then logically) escaped. Is there a way to prevent that? – AstroFloyd Aug 24 '20 at 15:35
  • @DannyBullis json has pprint built-in: json.dumps(x, default=str, indent=4) – Berry Tsakala Oct 19 '20 at 16:59
  • @Mark `"but not so great when deserializing"` - it returns a standartized parseable string. Do you know better human-readable json compatible representation? – Smit Johnth Jan 12 '21 at 03:42
  • @SmitJohnth I'm glad you ask, because this is why I made json-tricks: https://github.com/mverleg/pyjson_tricks – Mark Jan 15 '21 at 21:49
  • man, this should be the default in Python – orcaman Jan 20 '21 at 09:56
  • Django users: https://docs.djangoproject.com/en/3.2/topics/serialization/#djangojsonencoder – Vikram Ray Nov 15 '21 at 12:02
  • 1
    I have to wonder why default does not default to str in the first place. – AlanObject Dec 19 '22 at 18:35
602

Building on other answers, a simple solution based on a specific serializer that just converts datetime.datetime and datetime.date objects to strings.

from datetime import date, datetime

def json_serial(obj):
    """JSON serializer for objects not serializable by default json code"""

    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError ("Type %s not serializable" % type(obj))

As seen, the code just checks to find out if object is of class datetime.datetime or datetime.date, and then uses .isoformat() to produce a serialized version of it, according to ISO 8601 format, YYYY-MM-DDTHH:MM:SS (which is easily decoded by JavaScript). If more complex serialized representations are sought, other code could be used instead of str() (see other answers to this question for examples). The code ends by raising an exception, to deal with the case it is called with a non-serializable type.

This json_serial function can be used as follows:

from datetime import datetime
from json import dumps

print dumps(datetime.now(), default=json_serial)

The details about how the default parameter to json.dumps works can be found in Section Basic Usage of the json module documentation.

Jay Taylor
  • 13,185
  • 11
  • 60
  • 85
jgbarah
  • 7,334
  • 1
  • 20
  • 23
  • 8
    yeah the correct answer, more pretty import datetime and if isinstance(obj, datetime.datetime) , I lost many time because don't used from datetime import datetime , anyway thanks – Sérgio Sep 22 '14 at 20:43
  • 18
    but this does not explain how to deserialise it with the correct type, isnt it ? – BlueTrin Jun 26 '15 at 15:53
  • 4
    No, @BlueTrin , nothing said about that. In my case, I'm deserializing in JavaScript, which works out of the box. – jgbarah Jun 28 '15 at 08:09
  • 2
    This will cause unexpected behavior if the json module ever updates to include serialization of datetime objects. – Justin Aug 16 '16 at 17:27
  • I'm worried that the error messages this replaces will be less helpful, and that it woudln't work on non-datatime values at all. I don't fully understand the api this working with. All I expect is that you'd have to call the base class in the default case like http://stackoverflow.com/a/27058505/1695680 does. – ThorSummoner Sep 27 '16 at 22:43
  • Keep in mind that `obj.isoformat()` (and `str(obj)`) will output the timezone info (like `-08:00`), so if the same date was defined in different time zones their json representations won't match. The solution would be to convert dates to either UTC or timestamps. – serg Sep 06 '17 at 18:00
  • "If specified, default should be a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a TypeError. If not specified, TypeError is raised." Thanks. – WesternGun Apr 02 '18 at 09:53
  • 2
    @serg But converting times to UTC would unify `01:00:00+01:00` and `02:00:00+00:00` which are not supposed to be the same, depending on context. They refer to the same point in time of course, but the offset might be a relevant aspect of the value. – Alfe Sep 18 '19 at 12:26
  • +1 for supporting `datetime.date`. However, maybe even better idea would be to use duck typing instead of isinstance, to support all `obj` that implements `isoformat()` method (e.g. pendulum and arrow)? – Toreno96 May 19 '20 at 10:11
  • How would you do that in a nested dictionary that somewhere has datetime? – elano7 Aug 30 '22 at 13:21
545

Updated for 2018

The original answer accommodated the way MongoDB "date" fields were represented as:

{"$date": 1506816000000}

If you want a generic Python solution for serializing datetime to json, check out @jjmontes' answer for a quick solution which requires no dependencies.


As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:
http://api.mongodb.org/python/1.10.1/api/bson/json_util.html

Example usage (serialization):

from bson import json_util
import json

json.dumps(anObject, default=json_util.default)

Example usage (deserialization):

json.loads(aJsonString, object_hook=json_util.object_hook)

Django

Django provides a native DjangoJSONEncoder serializer that deals with this kind of properly.

See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder

from django.core.serializers.json import DjangoJSONEncoder

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  cls=DjangoJSONEncoder
)

One difference I've noticed between DjangoJSONEncoder and using a custom default like this:

import datetime
import json

def default(o):
    if isinstance(o, (datetime.date, datetime.datetime)):
        return o.isoformat()

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  default=default
)

Is that Django strips a bit of the data:

 "last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder 
 "last_login": "2018-08-03T10:51:42.990239", # default

So, you may need to be careful about that in some cases.

Aaron Lelevier
  • 19,850
  • 11
  • 76
  • 111
jdi
  • 90,542
  • 19
  • 167
  • 203
  • 5
    Is it good/bad practice to be mixing multiple libraries i.e. having mongoengine for inserting docs and pymongo for query/retrieval? – Rolando Aug 09 '12 at 02:25
  • Its not bad practice, it just implies some dependency on the libraries that your main library uses. If you can't accomplish what you need from mongoengine, then you drop down to pymongo. Its the same with `Django MongoDB`. With the later, you would try to stay within the django ORM to maintain backend agnostic state. But sometimes you can't do what you need in the abstraction, so you drop down a layer. In this case, its completely unrelated to your problem since you are just using utility methods to accompany the JSON format. – jdi Aug 09 '12 at 02:29
  • I am trying this out with Flask and it appears that by using json.dump, I am unable to put a jsonify() wrapper around it such that it returns in application/json. Attempting to do return jsonify(json.dumps(sample, default=json_util.default)) – Rolando Aug 09 '12 at 02:39
  • I am assuming `jsonify` would try and serialize it again. Is it a flask function specifically? – jdi Aug 09 '12 at 02:41
  • I think it is a flask function specifically to get it to return of "content-type: application/json" that usually accepts a dict, but I get: "ValueError: dictionary update sequence element #0 has length 1; 2 is required". I hope django is not better for this sort of thing... – Rolando Aug 09 '12 at 02:43
  • You would just need to use the `flask` general response object and set the content-type – jdi Aug 09 '12 at 02:43
  • 5
    @amit Its not so much about memorizing syntax, as it is about getting good at reading documentation and storing enough info in my head to recognise where and when I need to retrieve it again. In this case, one might say "Oh a custom object with json" and then quickly refresh on that usage – jdi Jul 02 '15 at 19:12
  • I had been using this method `bson.json_util.dumps(anObject)`, but your example method `json.dumps(anObject, default=bson.json_util.default)` is exponentially faster. Thank you! – Rob.Kachmar Mar 24 '17 at 19:59
  • This will convert datetime to a dict with $date, not I wanted: `json.dumps({'d':d}, default=json_util.default)` -> `'{"d": {"$date": 1506816000000}}'` – guyskk Nov 07 '17 at 10:13
  • 2
    @guyskk I haven't tracked changes in bjson or mongo since I wrote this 5 years ago. But if you want control over the serialisation of the datetime then you need to write your own default handler function as illustrated in the answer given by jgbarah – jdi Nov 07 '17 at 10:24
  • The answer by jgbarah doesn't have any example for the object_hook, and the bjson source code doesn't show any sign of `json_util` or `object_hook`, 5 years ago or today. Can anyone point me to an example? I fundamentally don't know input/return structure is expected except for just guessing and checking. – AlanSE Mar 08 '18 at 15:55
  • @AlanSE the bson module is being provided by pymongo, so you can see the source here: https://github.com/mongodb/mongo-python-driver/blob/master/bson/json_util.py – jdi Mar 08 '18 at 18:22
  • in the personal don't like use external librarys, i prefer the utisl from python as example datetime, isoformat()..etc.. – Diego Avila Apr 23 '18 at 18:55
  • @DiegoAvila sure under normal circumstances it is wise to always evaluate your dependencies. But in this case the question was about the Python mongodb client which comes with the needed serialisation facilities. – jdi Apr 23 '18 at 19:19
  • Also in Django, if you have this problem with model serialization in a JSONField (from django.contrib.postgres.fields), the solution is to add encoder=DjangoJSONEncoder. ie: my_field = JSONField(... encoder=DjangoJSONEncoder) – mattdedek Feb 27 '19 at 04:37
  • The question is not tagged as either Mongo or Django...but low and behold. – aaa90210 Jun 26 '19 at 06:02
  • @aaa90210 yea thought the immediate questions that followed in the comments clarified that it was a mongodb and Django situation – jdi Jun 27 '19 at 08:49
  • When I am using json.loads(aJsonString, object_hook=json_util.object_hook) I am getting Object of type ObjectId is not JSON serializable – Nilay Singh Feb 08 '21 at 11:42
  • @NilaySingh that sounds similar to this other question that I answered. Might want to check your json string format and make sure it uses "$oid" and not "ObjectId()" in the string form: https://stackoverflow.com/a/8409539/496445 – jdi Feb 09 '21 at 18:29
  • @jdi I have a complex json object with a timedelta format and it is only working with `DjangoJSONEncoder` I am not sure if I can any other default ways or not. What do you think? – Moustafa Mahmoud Oct 19 '21 at 12:31
287

I have just encountered this problem and my solution is to subclass json.JSONEncoder:

from datetime import datetime
import json

class DateTimeEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, datetime):
            return o.isoformat()

        return json.JSONEncoder.default(self, o)

In your call do something like: json.dumps(yourobj, cls=DateTimeEncoder) The .isoformat() I got from one of the answers above.

lenny
  • 2,941
  • 1
  • 14
  • 6
144

Convert the date to a string

sample['somedate'] = str( datetime.utcnow() )
Nam G VU
  • 33,193
  • 69
  • 233
  • 372
D.A
  • 2,555
  • 1
  • 14
  • 10
  • 11
    And how could I deserialize it in Python? – wobmene Mar 27 '14 at 12:58
  • 72
    The problem is if you have many datetime objects embedded deeply in a data structure, or they are random. This isn't a reliable method. – Rebs May 10 '14 at 06:37
  • 3
    to deserialize: `oDate = datetime.datetime.strptime(sDate, '%Y-%m-%d %H:%M:%S.%f')`. Formats obtained from: https://docs.python.org/2/library/datetime.html – Roman May 11 '15 at 10:45
  • 14
    Downvoted as it ignores timezone information. Keep in mind that `.now()` uses local time, without indicating this. At least `.utcnow()` should be used (and then a +0000 or Z appended) – Daniel F Aug 25 '15 at 20:06
  • 5
    @DanielF `At least .utcnow() should be used` Not exactly, `datetime.now(timezone.utc)` is recommended, see warning in: https://docs.python.org/3.8/library/datetime.html#datetime.datetime.utcnow. – Toreno96 May 19 '20 at 10:04
85

For others who do not need or want to use the pymongo library for this.. you can achieve datetime JSON conversion easily with this small snippet:

def default(obj):
    """Default JSON serializer."""
    import calendar, datetime

    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()
        millis = int(
            calendar.timegm(obj.timetuple()) * 1000 +
            obj.microsecond / 1000
        )
        return millis
    raise TypeError('Not sure how to serialize %s' % (obj,))

Then use it like so:

import datetime, json
print json.dumps(datetime.datetime.now(), default=default)

output: 

'1365091796124'
Vajk Hermecz
  • 5,413
  • 2
  • 34
  • 25
Jay Taylor
  • 13,185
  • 11
  • 60
  • 85
  • 1
    Shouldn't `millis=` be indented inside the if statement? It's also probably better to use str(obj) to get the ISO format which I would think is more common. – Rebs May 10 '14 at 06:45
  • Why would you want it to be indented? This snippet works and the resulting output can easily be deserialized/parsed from javascript. – Jay Taylor May 12 '14 at 00:29
  • 5
    Because obj may not be a [time, date, datetime] object – Rebs May 13 '14 at 02:30
  • 2
    your example is incorrect if the local timezone has non-zero UTC offset (most of them). `datetime.now()` returns local time (as a naive datetime object) but your code assumes that `obj` is in UTC if it is not timezone-aware. Use `datetime.utcnow()` instead. – jfs Oct 12 '14 at 23:25
  • 2
    Adjusted it to raise a type error if obj is unrecognized as per the Python documentation recommendation at https://docs.python.org/2/library/json.html#basic-usage. – Jay Taylor Nov 05 '15 at 18:27
69

Here is my solution:

import json


class DatetimeEncoder(json.JSONEncoder):
    def default(self, obj):
        try:
            return super().default(obj)
        except TypeError:
            return str(obj)

Then you can use it like that:

json.dumps(dictionnary, cls=DatetimeEncoder)
Natim
  • 17,274
  • 23
  • 92
  • 150
64

if you are using python3.7, then the best solution is using datetime.isoformat() and datetime.fromisoformat(); they work with both naive and aware datetime objects:

#!/usr/bin/env python3.7

from datetime import datetime
from datetime import timezone
from datetime import timedelta
import json

def default(obj):
    if isinstance(obj, datetime):
        return { '_isoformat': obj.isoformat() }
    raise TypeError('...')

def object_hook(obj):
    _isoformat = obj.get('_isoformat')
    if _isoformat is not None:
        return datetime.fromisoformat(_isoformat)
    return obj

if __name__ == '__main__':
    #d = { 'now': datetime(2000, 1, 1) }
    d = { 'now': datetime(2000, 1, 1, tzinfo=timezone(timedelta(hours=-8))) }
    s = json.dumps(d, default=default)
    print(s)
    print(d == json.loads(s, object_hook=object_hook))

output:

{"now": {"_isoformat": "2000-01-01T00:00:00-08:00"}}
True

if you are using python3.6 or below, and you only care about the time value (not the timezone), then you can use datetime.timestamp() and datetime.fromtimestamp() instead;

if you are using python3.6 or below, and you do care about the timezone, then you can get it via datetime.tzinfo, but you have to serialize this field by yourself; the easiest way to do this is to add another field _tzinfo in the serialized object;

finally, beware of precisions in all these examples;

Cyker
  • 9,946
  • 8
  • 65
  • 93
  • datetime.isoformat() is also present in Python 2.7: https://docs.python.org/2/library/datetime.html#datetime.datetime.isoformat – powlo Jun 10 '19 at 08:56
  • Very helpful solution. Could it also support `datetime.time`? – idbrii Jul 17 '21 at 23:16
  • 1
    @idbrii As @egvo said, you forgot `default=default` in your edit, which introduced the error. But you are right `super().default(obj)` is useless here. In fact, it came from a json-encoder example where it is useful. – Cyker Jul 28 '21 at 18:33
  • 1
    this is my favorite answer, since it a) includes deserialization and b) is very simple/easy to understand – Erik Aronesty Mar 13 '23 at 16:00
32

You should apply .strftime() method on .datetime.now() method to make it as a serializable method.

Here's an example:

from datetime import datetime

time_dict = {'time': datetime.now().strftime('%Y-%m-%dT%H:%M:%S')}
sample_dict = {'a': 1, 'b': 2}
sample_dict.update(time_dict)
sample_dict

Output:

Out[0]: {'a': 1, 'b': 2, 'time': '2017-10-31T15:16:30'}

[UPDATE]:

In Python3.7 and later, you can simply use the .isoformat() method:

from datetime import datetime

datetime.now().isoformat()
Benyamin Jafari
  • 27,880
  • 26
  • 135
  • 150
24

The json.dumps method can accept an optional parameter called default which is expected to be a function. Every time JSON tries to convert a value it does not know how to convert it will call the function we passed to it. The function will receive the object in question, and it is expected to return the JSON representation of the object.

def myconverter(o):
 if isinstance(o, datetime.datetime):
    return o.__str__()

print(json.dumps(d, default = myconverter)) 
Saurabh Saha
  • 962
  • 11
  • 18
22

I have an application with a similar issue; my approach was to JSONize the datetime value as a 6-item list (year, month, day, hour, minutes, seconds); you could go to microseconds as a 7-item list, but I had no need to:

class DateTimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            encoded_object = list(obj.timetuple())[0:6]
        else:
            encoded_object =json.JSONEncoder.default(self, obj)
        return encoded_object

sample = {}
sample['title'] = "String"
sample['somedate'] = datetime.datetime.now()

print sample
print json.dumps(sample, cls=DateTimeEncoder)

produces:

{'somedate': datetime.datetime(2013, 8, 1, 16, 22, 45, 890000), 'title': 'String'}
{"somedate": [2013, 8, 1, 16, 22, 45], "title": "String"}
codingatty
  • 2,026
  • 1
  • 23
  • 32
21

My solution (with less verbosity, I think):

def default(o):
    if type(o) is datetime.date or type(o) is datetime.datetime:
        return o.isoformat()

def jsondumps(o):
    return json.dumps(o, default=default)

Then use jsondumps instead of json.dumps. It will print:

>>> jsondumps({'today': datetime.date.today()})
'{"today": "2013-07-30"}'

I you want, later you can add other special cases to this with a simple twist of the default method. Example:

def default(o):
    if type(o) is datetime.date or type(o) is datetime.datetime:
        return o.isoformat()
    if type(o) is decimal.Decimal:
        return float(o)
fiatjaf
  • 11,479
  • 5
  • 56
  • 72
  • 1
    You should use isinstance(o, (datetime.date, datetime.datetime,)). Probably wouldn't hurt to include datetime.time too. – Rebs May 10 '14 at 06:56
  • I don't think this is a good solution anymore. Probably the conversions should take a more privileged place -- and also a more understandable place -- in your code, so you know what you are converting to when you put things into a database, or whatever, instead of having everything being done by a transparent function. But I don't know. – fiatjaf May 10 '14 at 11:31
  • 2
    JSON is good for serialising data for processing later. You may not know exactly what that data is. And you shouldn't need to. Serialising JSON should just work. Just like converting unicode to ascii should. Python's inability to do this without obscure functions makes it annoying to use. Database validation is a separate issue IMO. – Rebs May 11 '14 at 11:52
  • No, it shouldn't "just work". If you don't know how the serialization occurred and has to access the data later from another program/language, then you're lost. – fiatjaf May 12 '14 at 01:45
  • 2
    JSON is commonly used for strings, ints, floats, dates (I'm sure others use currency, temperatures, commonly too). But datetime is part of the standard library and should support de/serialisation. If it wasn't for this question, I'd still be manually searching my incredibly complex json blobs (which I didn't always create the structure for) for dates and serialising them 1 by 1. – Rebs May 13 '14 at 03:11
  • @AdamGriffiths: what should the serialized value be? How do you serialize `datetime(2014, 10, 13, 19, 31, 0, 123456, tzinfo=)` without losing info? `.timestamp()` -- `1413243060.123456` is efficient and easy to parse but the original UTC offset is lost, `.isoformat()` -- `2014-10-13T19:31:00.123456-04:00` is better but it loses zoneinfo (UTC offset along is ambiguous if you want to do date arithmetic, or would you combine them: `{'rfc3339': '2014-10-13T19:31:00.123456-04:00', 'timezone': 'America/New_York', 'timestamp': 1413243060.123456}`? – jfs Oct 12 '14 at 23:43
  • Point taken. Dates are messy. I'm starting to agree with @fiatjaf, but I still believe part of the problem lies in the python standard library. – Rebs Oct 14 '14 at 03:04
19

This Q repeats time and time again - a simple way to patch the json module such that serialization would support datetime.

import json
import datetime

json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)

Than use json serialization as you always do - this time with datetime being serialized as isoformat.

json.dumps({'created':datetime.datetime.now()})

Resulting in: '{"created": "2015-08-26T14:21:31.853855"}'

See more details and some words of caution at: StackOverflow: JSON datetime between Python and JavaScript

Community
  • 1
  • 1
davidhadas
  • 2,333
  • 21
  • 17
  • Monkey patch FTW. The nasty thing is of course that this modifies the behaviour of the json module in your whole application, which may surprise others in a large application, so should generally be used with care imho. – Jaap Versteegh Feb 10 '19 at 19:16
11

You have to supply a custom encoder class with the cls parameter of json.dumps. To quote from the docs:

>>> import json
>>> class ComplexEncoder(json.JSONEncoder):
...     def default(self, obj):
...         if isinstance(obj, complex):
...             return [obj.real, obj.imag]
...         return json.JSONEncoder.default(self, obj)
...
>>> dumps(2 + 1j, cls=ComplexEncoder)
'[2.0, 1.0]'
>>> ComplexEncoder().encode(2 + 1j)
'[2.0, 1.0]'
>>> list(ComplexEncoder().iterencode(2 + 1j))
['[', '2.0', ', ', '1.0', ']']

This uses complex numbers as the example, but you can just as easily create a class to encode dates (except I think JSON is a little fuzzy about dates)

Sean Redmond
  • 3,974
  • 22
  • 28
11

Here is a simple solution to over come "datetime not JSON serializable" problem.

enco = lambda obj: (
    obj.isoformat()
    if isinstance(obj, datetime.datetime)
    or isinstance(obj, datetime.date)
    else None
)

json.dumps({'date': datetime.datetime.now()}, default=enco)

Output:-> {"date": "2015-12-16T04:48:20.024609"}

ob92
  • 111
  • 1
  • 4
7

The simplest way to do this is to change the part of the dict that is in datetime format to isoformat. That value will effectively be a string in isoformat which json is ok with.

v_dict = version.dict()
v_dict['created_at'] = v_dict['created_at'].isoformat()
Peter Graham
  • 2,467
  • 2
  • 24
  • 29
7

Actually it is quite simple. If you need to often serialize dates, then work with them as strings. You can easily convert them back as datetime objects if needed.

If you need to work mostly as datetime objects, then convert them as strings before serializing.

import json, datetime

date = str(datetime.datetime.now())
print(json.dumps(date))
"2018-12-01 15:44:34.409085"
print(type(date))
<class 'str'>

datetime_obj = datetime.datetime.strptime(date, '%Y-%m-%d %H:%M:%S.%f')
print(datetime_obj)
2018-12-01 15:44:34.409085
print(type(datetime_obj))
<class 'datetime.datetime'>

As you can see, the output is the same in both cases. Only the type is different.

AngelDown
  • 291
  • 2
  • 4
6

Try this one with an example to parse it:

#!/usr/bin/env python

import datetime
import json

import dateutil.parser  # pip install python-dateutil


class JSONEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()
        return super(JSONEncoder, self).default(obj)


def test():
    dts = [
        datetime.datetime.now(),
        datetime.datetime.now(datetime.timezone(-datetime.timedelta(hours=4))),
        datetime.datetime.utcnow(),
        datetime.datetime.now(datetime.timezone.utc),
    ]
    for dt in dts:
        dt_isoformat = json.loads(json.dumps(dt, cls=JSONEncoder))
        dt_parsed = dateutil.parser.parse(dt_isoformat)
        assert dt == dt_parsed
        print(f'{dt}, {dt_isoformat}, {dt_parsed}')
        # 2018-07-22 02:22:42.910637, 2018-07-22T02:22:42.910637, 2018-07-22 02:22:42.910637
        # 2018-07-22 02:22:42.910643-04:00, 2018-07-22T02:22:42.910643-04:00, 2018-07-22 02:22:42.910643-04:00
        # 2018-07-22 06:22:42.910645, 2018-07-22T06:22:42.910645, 2018-07-22 06:22:42.910645
        # 2018-07-22 06:22:42.910646+00:00, 2018-07-22T06:22:42.910646+00:00, 2018-07-22 06:22:42.910646+00:00


if __name__ == '__main__':
    test()
zhigang
  • 6,597
  • 4
  • 30
  • 24
4

I usually use orjson. Not only because of its tremendous performance, but also for its great (RFC-3339 compliant) support of datetime:

import orjson # via pip3 install orjson
from datetime import datetime

data = {"created_at": datetime(2022, 3, 1)}

orjson.dumps(data) # returns b'{"created_at":"2022-03-01T00:00:00"}'

If you would like to use datetime.datetime objects without a tzinfo as UTC you can add the related option:

orjson.dumps(data, option=orjson.OPT_NAIVE_UTC) # returns b'{"created_at":"2022-03-01T00:00:00+00:00"}'
Michael Dorner
  • 17,587
  • 13
  • 87
  • 117
  • 1
    This is a great answer. No idea why the native `json` module can't handle datetime when virtually all other libraries in other languages do. This should be the standard. – adelriosantiago Aug 28 '23 at 20:34
4

If you are working with django models you can directly pass encoder=DjangoJSONEncoder to the field constructor. It will work like a charm.

from django.core.serializers.json import DjangoJSONEncoder 
from django.db import models 
from django.utils.timezone import now


class Activity(models.Model):
    diff = models.JSONField(null=True, blank=True, encoder=DjangoJSONEncoder)


diff = {
    "a": 1,
    "b": "BB",
    "c": now()
}

Activity.objects.create(diff=diff)
michal-michalak
  • 827
  • 10
  • 6
3

Generally there are several ways to serialize datetimes, like:

  1. ISO string, short and can include timezone info, e.g. @jgbarah's answer
  2. Timestamp (timezone data is lost), e.g. @JayTaylor's answer
  3. Dictionary of properties (including timezone).

If you're okay with the last way, the json_tricks package handles dates, times and datetimes including timezones.

from datetime import datetime
from json_tricks import dumps
foo = {'title': 'String', 'datetime': datetime(2012, 8, 8, 21, 46, 24, 862000)}
dumps(foo)

which gives:

{"title": "String", "datetime": {"__datetime__": null, "year": 2012, "month": 8, "day": 8, "hour": 21, "minute": 46, "second": 24, "microsecond": 862000}}

So all you need to do is

`pip install json_tricks`

and then import from json_tricks instead of json.

The advantage of not storing it as a single string, int or float comes when decoding: if you encounter just a string or especially int or float, you need to know something about the data to know if it's a datetime. As a dict, you can store metadata so it can be decoded automatically, which is what json_tricks does for you. It's also easily editable for humans.

Disclaimer: it's made by me. Because I had the same problem.

Community
  • 1
  • 1
Mark
  • 18,730
  • 7
  • 107
  • 130
2

If you are using the result in a view be sure to return a proper response. According to the API, jsonify does the following:

Creates a Response with the JSON representation of the given arguments with an application/json mimetype.

To mimic this behavior with json.dumps you have to add a few extra lines of code.

response = make_response(dumps(sample, cls=CustomEncoder))
response.headers['Content-Type'] = 'application/json'
response.headers['mimetype'] = 'application/json'
return response

You should also return a dict to fully replicate jsonify's response. So, the entire file will look like this

from flask import make_response
from json import JSONEncoder, dumps


class CustomEncoder(JSONEncoder):
    def default(self, obj):
        if set(['quantize', 'year']).intersection(dir(obj)):
            return str(obj)
        elif hasattr(obj, 'next'):
            return list(obj)
        return JSONEncoder.default(self, obj)

@app.route('/get_reps/', methods=['GET'])
def get_reps():
    sample = ['some text', <datetime object>, 123]
    response = make_response(dumps({'result': sample}, cls=CustomEncoder))
    response.headers['Content-Type'] = 'application/json'
    response.headers['mimetype'] = 'application/json'
    return response
Nick Cox
  • 35,529
  • 6
  • 31
  • 47
reubano
  • 5,087
  • 1
  • 42
  • 41
  • 1
    Question has nothing to do with flask. – Zoran Pavlovic Sep 09 '14 at 21:16
  • 3
    The question is about python. My answer solves the question using python. The OP did not say if the solution should include or exclude certain libraries. It is also useful for anyone else reading this question who wants an alternative to `pymongo`. – reubano Sep 10 '14 at 07:24
  • They question is both about Python and *not* about Flask. Flask is not even needed in your answer to the question, so I suggest you remove it. – Zoran Pavlovic Sep 10 '14 at 10:37
  • 1
    In flask it's much easier to use `flask.json.dumps` it handles datetime objects. – Jonatan Jul 31 '20 at 10:10
2

Here is my full solution for converting datetime to JSON and back..

import calendar, datetime, json

def outputJSON(obj):
    """Default JSON serializer."""

    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()

        return obj.strftime('%Y-%m-%d %H:%M:%S.%f')
    return str(obj)

def inputJSON(obj):
    newDic = {}

    for key in obj:
        try:
            if float(key) == int(float(key)):
                newKey = int(key)
            else:
                newKey = float(key)

            newDic[newKey] = obj[key]
            continue
        except ValueError:
            pass

        try:
            newDic[str(key)] = datetime.datetime.strptime(obj[key], '%Y-%m-%d %H:%M:%S.%f')
            continue
        except TypeError:
            pass

        newDic[str(key)] = obj[key]

    return newDic

x = {'Date': datetime.datetime.utcnow(), 34: 89.9, 12.3: 90, 45: 67, 'Extra': 6}

print x

with open('my_dict.json', 'w') as fp:
    json.dump(x, fp, default=outputJSON)

with open('my_dict.json') as f:
    my_dict = json.load(f, object_hook=inputJSON)

print my_dict

Output

{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}
{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}

JSON File

{"Date": "2013-11-08 02:30:56.479727", "34": 89.9, "45": 67, "12.3": 90, "Extra": 6}

This has enabled me to import and export strings, ints, floats and datetime objects. It shouldn't be to hard to extend for other types.

Hovo
  • 455
  • 4
  • 9
  • 1
    It explodes in Python 3 with `TypeError: 'str' does not support the buffer interface`. It is because of `'wb'` open mode, should be `'w'`. It also blows in deserialization when we have data similar to date like `'0000891618-05-000338'` but not matching pattern. – omikron Feb 14 '14 at 16:14
2

My solution ...

from datetime import datetime
import json

from pytz import timezone
import pytz


def json_dt_serializer(obj):
    """JSON serializer, by macm.
    """
    rsp = dict()
    if isinstance(obj, datetime):
        rsp['day'] = obj.day
        rsp['hour'] = obj.hour
        rsp['microsecond'] = obj.microsecond
        rsp['minute'] = obj.minute
        rsp['month'] = obj.month
        rsp['second'] = obj.second
        rsp['year'] = obj.year
        rsp['tzinfo'] = str(obj.tzinfo)
        return rsp
    raise TypeError("Type not serializable")


def json_dt_deserialize(obj):
    """JSON deserialize from json_dt_serializer, by macm.
    """
    if isinstance(obj, str):
        obj = json.loads(obj)
    tzone = timezone(obj['tzinfo'])
    tmp_dt = datetime(obj['year'],
                      obj['month'],
                      obj['day'],
                      hour=obj['hour'],
                      minute=obj['minute'],
                      second=obj['second'],
                      microsecond=obj['microsecond'])
    loc_dt = tzone.localize(tmp_dt)
    deserialize = loc_dt.astimezone(tzone)
    return deserialize    

Ok, now some tests.

# Tests
now = datetime.now(pytz.utc)

# Using this solution
rsp = json_dt_serializer(now)
tmp = json_dt_deserialize(rsp)
assert tmp == now
assert isinstance(tmp, datetime) == True
assert isinstance(now, datetime) == True

# using default from json.dumps
tmp = json.dumps(datetime.now(pytz.utc), default=json_dt_serializer)
rsp = json_dt_deserialize(tmp)
assert isinstance(rsp, datetime) == True

# Lets try another timezone
eastern = timezone('US/Eastern')
now = datetime.now(eastern)
rsp = json_dt_serializer(now)
tmp = json_dt_deserialize(rsp)

print(tmp)
# 2015-10-22 09:18:33.169302-04:00

print(now)
# 2015-10-22 09:18:33.169302-04:00

# Wow, Works!
assert tmp == now
macm
  • 1,959
  • 23
  • 25
2

Convert the date to string

date = str(datetime.datetime(somedatetimehere)) 
PKirby
  • 859
  • 3
  • 16
  • 36
2

As per the @jjmontes answer, I have used the following approach. For flask and flask-restful users

# get json string
jsonStr = json.dumps(my_dictionary, indent=1, sort_keys=True, default=str)
# then covert json string to json object
return json.loads(jsonStr)
Hashan Shalitha
  • 835
  • 8
  • 15
1

I got the same error message while writing the serialize decorator inside a Class with sqlalchemy. So instead of :

Class Puppy(Base):
    ...
    @property
    def serialize(self):
        return { 'id':self.id,
                 'date_birth':self.date_birth,
                  ...
                }

I simply borrowed jgbarah's idea of using isoformat() and appended the original value with isoformat(), so that it now looks like:

                  ...
                 'date_birth':self.date_birth.isoformat(),
                  ...
Treefish Zhang
  • 1,131
  • 1
  • 15
  • 27
1

A quick fix if you want your own formatting

for key,val in sample.items():
    if isinstance(val, datetime):
        sample[key] = '{:%Y-%m-%d %H:%M:%S}'.format(val) #you can add different formating here
json.dumps(sample)
Mauro Baraldi
  • 6,346
  • 2
  • 32
  • 43
Arash
  • 609
  • 6
  • 10
1

If you are on both sides of the communication you can use repr() and eval() functions along with json.

import datetime, json

dt = datetime.datetime.now()
print("This is now: {}".format(dt))

dt1 = json.dumps(repr(dt))
print("This is serialised: {}".format(dt1))

dt2 = json.loads(dt1)
print("This is loaded back from json: {}".format(dt2))

dt3 = eval(dt2)
print("This is the same object as we started: {}".format(dt3))

print("Check if they are equal: {}".format(dt == dt3))

You shouldn't import datetime as

from datetime import datetime

since eval will complain. Or you can pass datetime as a parameter to eval. In any case this should work.

0

I had encountered same problem when externalizing django model object to dump as JSON. Here is how you can solve it.

def externalize(model_obj):
  keys = model_obj._meta.get_all_field_names() 
  data = {}
  for key in keys:
    if key == 'date_time':
      date_time_obj = getattr(model_obj, key)
      data[key] = date_time_obj.strftime("%A %d. %B %Y")
    else:
      data[key] = getattr(model_obj, key)
  return data
naren
  • 14,611
  • 5
  • 38
  • 45
0
def j_serial(o):     # self contained
    from datetime import datetime, date
    return str(o).split('.')[0] if isinstance(o, (datetime, date)) else None

Usage of above utility:

import datetime
serial_d = j_serial(datetime.datetime.now())
if serial_d:
    print(serial_d)  # output: 2018-02-28 02:23:15
0

I faced this issue today, I found something called pickle. It's a built-in library for serializing python objects and also load it from a pickle file.

The only difference I found between pickle and json is pickle file is a binary file, where as json is a usual text file.

And It doesn't cause any issues with datetime objects.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Appaji Chintimi
  • 613
  • 2
  • 7
  • 19
0

Another approach is to adopt a concept from FEEL (the Friendly Enough Expression Language) defined in DMN (Decision Model Notation) - namely @strings. Any string starting with @" and ending with " is decoded separately with FEEL decoding. Of course the sender and the receiver have to agree to this convention, but ... the code below lets you encode lots of other things as well as dates, times, date/times, timedeltas. You can encode year/month durations and ranges (so long as you except a 4 element tuple of chr, expr, expr, chr as being a good representation of a range - where the two chrs are open/close brackets). So, @"P4Y2M" is a duration of 4 years and 2 months. @"P2DT5H" is a timedelta of 2 days and 4 hours, @"(2021-01-02 .. 2021-12-31)" is a year range.

The following code can be used to serialize and de-serialize @strings.

import datetime
import pySFeel

parser = pySFeel.SFeelParser()


def convertAtString(thisString):
    # Convert an @string
    (status, newValue) = parser.sFeelParse(thisString[2:-1])
    if 'errors' in status:
        return thisString
    else:
        return newValue


def convertIn(newValue):
    if isinstance(newValue, dict):
        for key in newValue:
            if isinstance(newValue[key], int):
                newValue[key] = float(newValue[key])
            elif isinstance(newValue[key], str) and (newValue[key][0:2] == '@"') and (newValue[key][-1] == '"'):
                newValue[key] = convertAtString(newValue[key])
            elif isinstance(newValue[key], dict) or isinstance(newValue[key], list):
                newValue[key] = convertIn(newValue[key])
    elif isinstance(newValue, list):
        for i in range(len(newValue)):
            if isinstance(newValue[i], int):
                newValue[i] = float(newValue[i])
            elif isinstance(newValue[i], str) and (newValue[i][0:2] == '@"') and (newValue[i][-1] == '"'):
                newValue[i] = convertAtString(newValue[i])
            elif isinstance(newValue[i], dict) or isinstance(newValue[i], list):
                newValue[i] = convertIn(newValue[i])
    elif isinstance(newValue, str) and (newValue[0:2] == '@"') and (newValue[-1] == '"'):
        newValue = convertAtString(newValue)
    return newValue


  def convertOut(thisValue):
      if isinstance(thisValue, datetime.date):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.datetime):
          return '@"' + thisValue.isoformat(sep='T') + '"'
      elif isinstance(thisValue, datetime.time):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.timedelta):
          sign = ''
          duration = thisValue.total_seconds()
          if duration < 0:
              duration = -duration
              sign = '-'
          secs = duration % 60
          duration = int(duration / 60)
          mins = duration % 60
          duration = int(duration / 60)
          hours = duration % 24
          days = int(duration / 24)
          return '@"%sP%dDT%dH%dM%fS"' % (sign, days, hours, mins, secs)
      elif isinstance(thisValue, bool):
          return thisValue:
      elif thisValue is None:
          return thisValue:
      elif isinstance(thisValue, int):
          sign = ''
          if thisValue < 0:
              thisValue = -thisValue
              sign = '-'
          years = int(thisValue / 12)
          months = (thisValue % 12)
          return '@"%sP%dY%dM"' % (sign, years, months)
      elif isinstance(thisValue, tuple) and (len(thisValue) == 4):
          (lowEnd, lowVal, highVal, highEnd) = thisValue
          return '@"' + lowEnd + str(lowVal) + ' .. ' + str(highVal) + highEnd
      elif thisValue is None:
          return 'null'
      elif isinstance(thisValue, dict):
          for item in thisValue:
              thisValue[item] = convertOut(thisValue[item])
          return thisValue
      elif isinstance(thisValue, list):
          for i in range(len(thisValue)):
              thisValue[i] = convertOut(thisValue[i])
          return thisValue
      else:
          return thisValue
-2

I may not 100% correct but, this is the simple way to do serialize

#!/usr/bin/python
import datetime,json

sampledict = {}
sampledict['a'] = "some string"
sampledict['b'] = datetime.datetime.now()

print sampledict   # output : {'a': 'some string', 'b': datetime.datetime(2017, 4, 15, 5, 15, 34, 652996)}

#print json.dumps(sampledict)

'''
output : 

Traceback (most recent call last):
  File "./jsonencodedecode.py", line 10, in <module>
    print json.dumps(sampledict)
  File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2017, 4, 15, 5, 16, 17, 435706) is not JSON serializable


'''

sampledict['b'] = datetime.datetime.now().strftime("%B %d, %Y %H:%M %p")

afterdump = json.dumps(sampledict)

print afterdump  #output : {"a": "some string", "b": "April 15, 2017 05:18 AM"}

print type(afterdump) #<type 'str'>


afterloads = json.loads(afterdump) 

print afterloads # output : {u'a': u'some string', u'b': u'April 15, 2017 05:18 AM'}


print type(afterloads) # output :<type 'dict'> 
Shiva
  • 467
  • 2
  • 4
  • 12