Why does json.dumps(list(np.arange(5))) fail while json.dumps(np.arange(5).tolist()) works

Question

I noticed this problem when a computer running Ubuntu was updated recently and the default version of Python changed to 2.7.

import json
import numpy as np

json.dumps(list(np.arange(5))) # Fails, throws a "TypeError: 0 is not JSON serializable"
json.dumps(np.arange(5).tolist()) # Works

Is there a difference between list() and the tolist() methd of a numpy array?

I am seeing a similar problem, where I cannot `json.dumps()` a np.int64 variable. However, it works for me in Python 2.7.9 but not in 3.4. — Thomas Arildsen, May 28 '15 at 08:32

phihag · Answer 1 · 2012-07-19T17:11:37.947

23

Because the elements of a NumPy array are not native ints, but of NUmPy's own types:

>>> type(np.arange(5)[0])
<type 'numpy.int64'>

You can use a custom JSONEncoder to support the ndarray type returned by arange:

import numpy as np
import json

class NumPyArangeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.ndarray):
            return obj.tolist() # or map(int, obj)
        return json.JSONEncoder.default(self, obj)

print(json.dumps(np.arange(5), cls=NumPyArangeEncoder))

edited Jul 19 '12 at 17:11

answered Jul 19 '12 at 13:31

phihag

278,196
72
453
469

great answer! can you please also look at [this](https://stackoverflow.com/questions/68154853/using-json-dumps-to-serialize-class-with-dictionary-with-ndarray) – user0193 Jun 28 '21 at 11:56

score 23 · Accepted Answer · answered Jul 19 '12 at 13:31

23

It looks like the tolist() method turns the numpy int32 (or whatever size you have) back into an int, which JSON knows what to do with:

>>> list(np.arange(5))
[0, 1, 2, 3, 4]
>>> type(list(np.arange(5)))
<type 'list'>
>>> type(list(np.arange(5))[0])
<type 'numpy.int32'>
>>> np.arange(5).tolist()
[0, 1, 2, 3, 4]
>>> type(np.arange(5).tolist())
<type 'list'>
>>> type(np.arange(5).tolist()[0])
<type 'int'>

As the docs say for tolist():

Return the array as a (possibly nested) list.

Return a copy of the array data as a (nested) Python list. Data items are converted to the nearest compatible Python type.

The last line makes the difference here.

answered Jul 19 '12 at 13:31

DSM

342,061
65
592
494

Do you know if this a recent change? The code used to work before the system got upgraded. – azeey Jul 19 '12 at 13:40
Not sure, I'm afraid-- not even sure if the change was in numpy (say a type renaming) or on the Python JSON side (maybe it used to try harder to deal with unknown types?) – DSM Jul 19 '12 at 14:57
[Simple solution](http://stackoverflow.com/questions/8230315/python-sets-are-not-json-serializable) by explicitly passing a [default handler](http://docs.python.org/2/library/json.html#json.dumps) for non-serializable objects. – 0 _ Aug 22 '13 at 05:41

score 1 · Answer 3 · answered Jul 19 '12 at 13:31

1

The problem is that with the first you don't get an int. You get a numpy.int64. That cannot be serialized.

answered Jul 19 '12 at 13:31

Stefano Borini

138,652
96
297
431

Why does json.dumps(list(np.arange(5))) fail while json.dumps(np.arange(5).tolist()) works

3 Answers3

Linked