Python to JSON Serialization fails on Decimal

Question

I have a python object which includes some decimals. This is causing the json.dumps() to break.

I got the following solution from SO (e.g. Python JSON serialize a Decimal object) but the recoomended solution still does not work. Python website - has the exact same answer.

Any suggestions how to make this work?

Thanks. Below is my code. It looks like the dumps() doesn't even go into the specialized encoder.

clayton@mserver:~/python> cat test1.py
import json, decimal

class DecimalEncoder(json.JSONEncoder):
        def _iterencode(self, o, markers=None):
                print "here we go o is a == ", type(o)
                if isinstance(o, decimal.Decimal):
                        print "woohoo! got a decimal"
                        return (str(o) for o in [o])
                return super(DecimalEncoder, self)._iterencode(o, markers)

z = json.dumps( {'x': decimal.Decimal('5.5')}, cls=DecimalEncoder )
print z
clayton@mserver:~/python> python test1.py
Traceback (most recent call last):
  File "test1.py", line 11, in <module>
    z = json.dumps( {'x': decimal.Decimal('5.5')}, cls=DecimalEncoder )
  File "/home/clayton/python/Python-2.7.3/lib/python2.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/clayton/python/Python-2.7.3/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/clayton/python/Python-2.7.3/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/home/clayton/python/Python-2.7.3/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('5.5') is not JSON serializable
clayton@mserver:~/python>

Both the second answer on the linked page and [the actual Python docs](http://docs.python.org/2/library/json.html#json.JSONEncoder) give the correct answer, which is to override `default`. It should be obvious that you shouldn't be overriding `_iterencode`, as the leading underscore implies that that is an internal-only method. — Daniel Roseman, Jun 06 '13 at 08:36
json doesn't know how to handle `Decimal` so one should extend JSON encoder for Python data structureshttp://docs.python.org/2/library/json.html#py-to-json-table — oleg, Jun 06 '13 at 08:39

Martijn Pieters · Accepted Answer · 2014-05-26T06:42:15.350

81

It is not (no longer) recommended you create a subclass; the json.dump() and json.dumps() functions take a default function:

def decimal_default(obj):
    if isinstance(obj, decimal.Decimal):
        return float(obj)
    raise TypeError

json.dumps({'x': decimal.Decimal('5.5')}, default=decimal_default)

Demo:

>>> def decimal_default(obj):
...     if isinstance(obj, decimal.Decimal):
...         return float(obj)
...     raise TypeError
... 
>>> json.dumps({'x': decimal.Decimal('5.5')}, default=decimal_default)
'{"x": 5.5}'

The code you found only worked on Python 2.6 and overrides a private method that is no longer called in later versions.

edited May 26 '14 at 06:42

answered Jun 06 '13 at 08:36

Martijn Pieters

1,048,767
296
4,058
3,343

1

shouldn't your `decimal_default' function return `float(str(obj))` or something like this? – oleg Jun 06 '13 at 08:48
@oleg: The OP code sample used str. You can return anything you like as long as it is JSON serializable. – Martijn Pieters Jun 06 '13 at 08:49
Why not make a subclass? Isn't it far less DRY to pass an argument on every call in every file (and/or error prone to just try and use it wherever you expect a Decimal might be in the input dict)? – Kyle Baker Jul 25 '16 at 17:20
1

@KyleBaker: you can still subclass `json.JSONEncoder` and implement a [`default()` method](https://docs.python.org/3/library/json.html#json.JSONEncoder.default) on that subclass. – Martijn Pieters Jul 25 '16 at 17:38
3

casting to `float` will lose accuracy; the duplicate contains approaches that lose none. – Antti Haapala -- Слава Україні Jan 15 '17 at 20:46
@AnttiHaapala: only if producing JSON strings instead of JSON numbers is acceptable and desired. – Martijn Pieters Jan 15 '17 at 20:55
@MartijnPieters argh, there is one fix, that works in <~3.5 :/ – Antti Haapala -- Слава Україні Jan 15 '17 at 21:50
@AnttiHaapala: Tesdal's fake float approach? Yeah, no, there was never any guarantee that floats would be converted to JSON notation with `repr()`. :-/ – Martijn Pieters Jan 15 '17 at 22:18
this solution is fine, but it only handles `decimal.Decimal` unexpected type. – Fabio Caccamo Jun 11 '19 at 08:59
@FabioCaccamo: which is what the question is *about*. The solution is trivially extended for other unexpected types. – Martijn Pieters Jun 11 '19 at 14:14
@FabioCaccamo: your edit on the other hand was entirely inappropriate. You can post your own answer with that solution if you like, but please don't alter mine. Take into account that `str()` conversions are not always the correct approach for serialisation. It is not a fits-all solution. – Martijn Pieters Jun 11 '19 at 14:16

score 22 · Answer 2 · answered Sep 16 '15 at 16:49

22

I can't believe that no one here talked about using simplejson, which supports deserialization of Decimal out of the box.

import simplejson
from decimal import Decimal

simplejson.dumps({"salary": Decimal("5000000.00")})
'{"salary": 5000000.00}'

simplejson.dumps({"salary": Decimal("1.1")+Decimal("2.2")-Decimal("3.3")})
'{"salary": 0.0}'

answered Sep 16 '15 at 16:49

FearlessFuture

2,250
4
19
25

1

Thanks for the intro to `simplejson` – Dinesh Sep 16 '15 at 20:45
2

It seems simplejson has it's days numbered though, at least as a default part of Django https://docs.djangoproject.com/en/1.8/internals/deprecation/#deprecation-removed-in-1-7 – wasabigeek Nov 29 '15 at 07:08
6

Are there any downsides using simplejson like this? Otherwise why so many answers go it the hard way? – benjaminz Mar 04 '16 at 05:36

score 22 · Answer 3 · answered Sep 27 '16 at 05:32

If you're using Django. There is a great class for Decimal and date fields:

https://docs.djangoproject.com/en/1.10/topics/serialization/#djangojsonencoder

To use it:

import json
from django.core.serializers.json import DjangoJSONEncoder

json.dumps(value, cls=DjangoJSONEncoder)

Python to JSON Serialization fails on Decimal

3 Answers3

Linked

Related