JSON serializer class - why json.dumps() works while json.dump() does not?

Question

I want to json.dump() a dictionary that has keys of datetime.date type, and to do so I made a serializer class to encode the date keys as strings. To my surprise it works with json.dumps(), but throws an error with json.dump(). At the end I was able to dump using a workaround, but I'd like to understand what happens here: why does the serializer class work with json.dumps() and does not work with json.dump()?

I tried writing a similar function to use with default= instead of cls=, tried rewriting this back and forth, tried to adapt some examples I found in the web, but I always ended up with the same error. I don't know what I'm missing here, even after reading the docs. Please enlighten me what do I need to change here to be able to serialize the dates and dump the dictionary.

Here is the MRE (Python 3.8.6):

import json
import datetime


class JSONSerializer(json.JSONEncoder):
    @staticmethod
    def convert_if_date(_date):
        if isinstance(_date, datetime.date):
            return _date.strftime('%Y-%m-%d')
        return _date

    def date_insensitive_encode(self, obj):
        if isinstance(obj, dict):
            return {self.convert_if_date(k): v for k, v in obj.items()}
        return obj

    def encode(self, obj):
        return super(JSONSerializer, self).encode(
            self.date_insensitive_encode(obj))


test_dict = {datetime.date(2022, 7, 10): 'OK'}

print(json.dumps(test_dict, cls=JSONSerializer))
# this prints: {"2022-07-11": "OK"}

with open('dump.json', 'w') as w:
    json.dump(test_dict, w, cls=JSONSerializer)
# but this throws a TypeError

Full error:

Traceback (most recent call last):
  File "E:/Progz/Python/AppKH/datacontrol/so_mre.py", line 28, in <module>
    json.dump(test_dict, w, cls=JSONSerializer)
  File "C:\Program Files\Python38\lib\json\__init__.py", line 179, in dump
    for chunk in iterable:
  File "C:\Program Files\Python38\lib\json\encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "C:\Program Files\Python38\lib\json\encoder.py", line 376, in _iterencode_dict
    raise TypeError(f'keys must be str, int, float, bool or None, '
TypeError: keys must be str, int, float, bool or None, not date

@akaAbdullahMateen I'm not sure this is quite what OP is asking — DeepSpace, Jul 10 '22 at 20:46
@DeepSpace I know, and this is why it is in comments, not in the answer section below. I just put it here so other people can quickly determine which does what without needing to google search them. — akaAbdullahMateen, Jul 10 '22 at 20:47
The problem is the `encode()` method of the base `json.JSONEncoder` class is only utilized when the `dumps()` method is called, but not when `dump()` is. I encountered this issue once a while ago and figured-out something to make my custom `JSONEncoder` work regardless of which one was used. I'll post an answer if what I did before can be applied to what you're doing… — martineau, Jul 10 '22 at 21:12
@akaAbdullahMateen: FWIW I think it's very improbable that your comment will help anyone — so suggest you delete it. — martineau, Jul 10 '22 at 21:16

martineau · Accepted Answer · 2022-07-10T22:40:39.693

From previous experience, I thought the encode() method of the base json.JSONEncoder class was only used the when the dumps() method was called, and not when dump() was, however that's apparently not the case (so my previous solution didn't apply)…but the good news is I was able to do what I think you want done only in a different, simpler way that works with both methods — and it only requires overriding the encode() method to do so.

from collections.abc import MutableMapping
import json
import datetime


class JSONSerializer(json.JSONEncoder):
    def encode(self, obj):
        # Convert dictionary keys that are datatime.dates into strings.
        if isinstance(obj, MutableMapping):
            for key in list(obj.keys()):
                if isinstance(key, datetime.date):
                    strkey = key.strftime('%Y-%m-%d')
                    obj[strkey] = obj.pop(key)

        return super().encode(obj)


test_dict = {datetime.date(2022, 7, 10): 'OK'}

print(json.dumps(test_dict, cls=JSONSerializer))  # -> {"2022-07-11": "OK"}

with open('dump.json', 'w') as w:
    json.dump(test_dict, w, cls=JSONSerializer)  # # Works now, too.

Neat. Will have to have a closer look at that MutableMapping to understand what happens under the hood, but everything I need to go on is here. Thanks a lot! — Plaargath, Jul 11 '22 at 08:28
`MutableMapping ` is just a fancy (but generic) way of saying `dict`-like. — martineau, Jul 11 '22 at 15:27

JSON serializer class - why json.dumps() works while json.dump() does not?

1 Answers1