5

How can I convert nested dictionary to nested defaultdict?

dic = {"a": {"aa": "xxx"}}
default = defaultdict(lambda: None, dic)
print(default["dummy_key"])  # return None
print(default["a"]["dummy_key"])  # KeyError
Guillaume Jacquenot
  • 11,217
  • 6
  • 43
  • 49
Maiko Ohkawa
  • 853
  • 2
  • 11
  • 28

1 Answers1

12

You need to either loop or recurse over the nested dictionary, through all of its levels.

Unless it's potentially ridiculously deep (as in hundreds of levels), or so wide that small performance factors make a difference, recursion is probably simplest here:

def defaultify(d):
    if not isinstance(d, dict):
        return d
    return defaultdict(lambda: None, {k: defaultify(v) for k, v in d.items()})

Or if you want it to work with all mappings, not just dicts, you could use collections.abc.Mapping instead of dict in your isinstance check.


Of course this is assuming you have a pure nested dict. If you've got, say, something you parsed from a typical JSON response, where there might be dicts with list values with dict elements, you have to handle the other possibilities too:

def defaultify(d):
    if isinstance(d, dict):
        return defaultdict(lambda: None, {k: defaultify(v) for k, v in d.items()})
    elif isinstance(d, list):
        return [defaultify(e) for e in d]
    else:
        return d

But if this actually is coming from JSON, it's probably better to just use your defaultdict as an object_pairs_hook while the JSON is being parsed, rather than parsing it to a dict and then converting it to a defaultdict later.

There's an example in the docs of using an OrderedDict in place of dict, but that won't quite work for us—unlike OrderedDict and dict, defaultdict can't just take an iterable of pairs as its only argument; it needs the default value factory first. So we can bind that in, using functools.partial:

d = json.loads(jsonstring, object_hook_pairs=partial(defaultdict, lambda: None))

And so on.

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • Thank you, It worked very well. Yes, I have list too, your advice is greatly helpful for me. Thank you very much. – Maiko Ohkawa Apr 25 '18 at 04:07
  • @MaikoOhkawa If you actually _are_ getting this from `json.loads` or similar, you might want to consider building it a `defaultdict` in the first place, instead of first building a `dict` and then converting it. For example, the [`json` docs](https://docs.python.org/3/library/json.html#encoders-and-decoders) explain how to make a decoder use `OrderedDict` instead of `dict` for every JSON object—you can do the same thing to make it use a `defaultdict`. – abarnert Apr 25 '18 at 04:18
  • Oh, thank you very much. I tried it but an error occured. json.loads('{"a": {"aa": "xxx"}}', object_pairs_hook=collections.defaultdict) # TypeError: first argument must be callable or None Maybe I'm wrong. – Maiko Ohkawa Apr 25 '18 at 04:52
  • @MaikoOhkawa You can't just use `defaultdict`, because it needs an extra first argument—but you can use, e.g., `functools.partial(defaultdict, lambda: None)`, or `lambda *args, **kw: defaultdict(lambda: None, *args, **kw)`. I'll edit this into the answer. – abarnert Apr 25 '18 at 05:15
  • 2
    This works perfectly, thanks so much! By the way, I submitted an edit for your last example to change `object_hook_pairs` to `object_pairs_hook`, got an error and spotted the mispelling. – bergonzzi Aug 06 '18 at 17:06