json.dumps messes up order

Question

I'm working with the json module creating a json file containing entries of the like

json.dumps({"fields": { "name": "%s", "city": "%s", "status": "%s", "country": "%s" }})

However, in the json-file created the fields are in the wrong order

{"fields": {"status": "%s", "city": "%s", "name": "%s", "country": "%s"}}

which is a problem because the substitions for the %s-strings are now incorrect.

How can I force the dumps function to keep the given order?

@Klaus D. Yet `OrderedDict`s do have it (see my [answer](http://stackoverflow.com/a/43640347/237105) below) — Antony Hatchkins, Apr 26 '17 at 17:00
I had the same issue with python2. Python 3 fixed it for me. — Manish Jain, Aug 27 '18 at 22:49
For posterity, python dictionaries are ordered as of 3.6: https://mail.python.org/pipermail/python-dev/2016-September/146327.html — Julian Irwin, Jul 22 '21 at 13:48

Eric O. Lebigot · Accepted Answer · 2019-01-30T12:26:24.880

Like the other answers correctly state, before Python 3.6, dictionaries are unordered.

That said, JSON is also supposed to have unordered mappings, so in principle it does not make much sense to store ordered dictionaries in JSON. Concretely, this means that upon reading a JSON object, the order of the returned keys can be arbitrary.

A good way of preserving the order of a mapping (like a Python OrderedDict) in JSON is therefore to output an array of (key, value) pairs that you convert back to an ordered mapping upon reading:

>>> from collections import OrderedDict
>>> import json
>>> d = OrderedDict([(1, 10), (2, 20)])                                         
>>> print d[2]
20
>>> json_format = json.dumps(d.items())                   
>>> print json_format  # Order maintained
[[1, 10], [2, 20]]
>>> OrderedDict(json.loads(json_format))  # Reading from JSON: works!
OrderedDict([(1, 10), (2, 20)])
>>> _[2]  # This works!
20

(Note the way the ordered dictionary is constructed from a list of (key, value) pairs: OrderedDict({1: 10, 2: 20}) would not work: its keys are not necessarily ordered as in the dictionary literal, since the literal creates a Python dictionary whose keys are unordered.)

PS: Starting with Python 3.1, the json modules offers a hook for automatically converting a list of pairs (like above) to something else like an OrderedDict.

Even though JSON is unordered, it would still be useful to have an ordered abstraction for it in the json library, for the same reasons that OrderedDict is useful. For instance, I want to output some JSON objects for both human and machine readability, and I want to preserve a particular key order for the human aspect. — John B, Mar 07 '16 at 19:19
@John B There's a mechanism for that, see [my answer](https://stackoverflow.com/a/43640347/237105) below. — Antony Hatchkins, May 25 '17 at 14:19
One reason to have consistent ordering is if you need to generate a JSON response and then build a hash for use in an ETag or a cache. In this case we don't need it to be in a specific order, but we do need it to be in a consistent and reproducible order. Each time you call it you should get the same string. json.dumps(d, sort_keys=True) will accomplish that. — Chris Sattinger, Nov 29 '17 at 13:30
Fly in the ointment: Python 3.6+ dictionaries *are* ordered. — Gringo Suave, Jan 29 '19 at 18:17

score 27 · Answer 2 · answered Apr 26 '17 at 16:59

27

You can choose OrderedDict to be used instead of an ordinary dict when creating a json object to remember the order of insertions:

>>> from collections import OrderedDict
>>> a = '{"fields": { "name": "%s", "city": "%s", "status": "%s", "country": "%s" }}'
>>> b = json.loads(a, object_pairs_hook=OrderedDict)
>>> json.dumps(b)
'{"fields": {"name": "%s", "city": "%s", "status": "%s", "country": "%s"}}'

answered Apr 26 '17 at 16:59

Antony Hatchkins

31,947
10
111
111

2

This is interesting. Is it guaranteed that `json.dumps()` preserves the order of the `OrderedDict` `b`? I can't find anything in the documentation. – Eric O. Lebigot Jun 01 '17 at 14:51
@EOL `items = dct.items()`, line 355 in Lib/json/encoder.py (python3.6). I can't think of a way or reason for json lib to scramble this order. – Antony Hatchkins Jun 01 '17 at 17:46
2

Me neither, but it would be nice to officially guarantee it in the documentation (for the current and future versions of the `json` module), especially since JSON itself does not have any concept of ordered set of key/value pairs (http://json.org). – Eric O. Lebigot Jun 02 '17 at 09:35
1

I created a documentation enhancement request: http://bugs.python.org/issue30550. – Eric O. Lebigot Jun 02 '17 at 09:48

ShacharSh · Answer 3 · 2015-05-11T10:38:43.543

15

This is a dictionary, and dictionaries don't keep order. You can use OrderedDict instead.

You could also add the sort_keys=False parameter:

json.dumps(values, sort_keys=False)

edited May 11 '15 at 10:38

answered May 10 '15 at 14:44

ShacharSh

500
3
15

1

This is good for writing JSON, but not for reading: JSON mappings have no order, i.e., the mapping that you read from JSON is not guaranteed to be ordered as in the JSON file. – Eric O. Lebigot May 10 '15 at 15:06
1

I guess you meant `dumps` instead of `loads`? In fact, `loads` does not have any `sort_keys` argument, by default. – Eric O. Lebigot May 11 '15 at 02:41
1

You are right. I meant 'dumps' and not 'loads'. Sorry for that. I will edit this – ShacharSh May 11 '15 at 10:38
3

sort_keys already defaults to False, so this doesn't change anything. json.dumps will not preserve the order of keys from the supplied OrderedDict – Chris Sattinger Nov 29 '17 at 13:27

score 2 · Answer 4 · edited May 11 '15 at 02:42

2

You cannot create an OrderedDict from a dict because order has already changed the moment you create a dictionary.So best way is to use tuples to create a OrderedDict

from collections import OrderedDict
import json

a = (("name","foo"),("city","bar"),("status","baz"),("country","my"))

b = OrderedDict(a)

c = {"fileds": b}

print json.dumps(c)
Output:
{"fileds": {"name": "foo", "city": "bar", "status": "baz", "country": "my"}}

edited May 11 '15 at 02:42

Eric O. Lebigot

91,433
48
218
260

answered May 10 '15 at 15:29

Ajay

5,267
2
23
30

2

JSON has no concept of ordered mapping, so dumping the OrderedDict `b` to JSON can change the key order, in principle. You must dump the `.items()` of the OrderedDict, as in my answer. Furthermore, there is no need for the indirection you added with the `c` variable: it has no effect (but to complicate the code). – Eric O. Lebigot May 11 '15 at 02:47

score 2 · Answer 5 · answered Mar 02 '21 at 02:40

Python 3.6.1:

Python 3.6.1 (default, Oct 10 2020, 20:16:48)
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import json
>>> json.dumps({'b': 1, 'a': 2})
'{"b": 1, "a": 2}'

Python 2.7.5:

Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import json
>>> json.dumps({'b': 1, 'a': 2})
'{"a": 2, "b": 1}'

json.dumps messes up order

5 Answers5

Linked

Related