How to Pythonically map content from one dict to another in a fail safe manner?

Question

I've got one dict from an api:

initial_dict = {
    "content": {
        "text":
    },
    "meta": {
        "title": "something",
        "created": "2016-03-04 15:30",
        "author": "Pete",
        "extra": {
            "a": 123,
            "b": 456
        }
    }
}

and I need to map this to another dict:

new_dict = {
    "content_text": initial_dict['content']['text'],
    "meta_title": initial_dict['meta']['title'],
    "meta_extras": {
        "time_related": {
            initial_dict['meta']['created']
        },
        "by": initial_dict['meta']['author']
    }
}

The problem is that not all fields are always in the initial_dict. I can of course wrap the whole creation of new_dict into a try/except, but then it would fail if one of the initial fields doesn't exist.

Is there no other way than creating a try/except for each and every field I add to the new_dict? In reality the dict is way bigger than this (about 400 key/value pairs), so this will become a mess quite fast.

Isn't there a better and more pythonic way of doing this?

Create a list where each entry is the from and to keys, loop through these in a single try/except — DisappointedByUnaccountableMod, Mar 09 '16 at 12:19
@barny - Sounds like a good idea, but I'm not sure how to construct something like that. Would you have some example code? — kramer65, Mar 09 '16 at 12:20
You could use `initial_dict.get('meta', dict()).get(key)`. By defaulting to `dict()` on the first call to `get`, you will remove the possibility of an error occurring — gtlambert, Mar 09 '16 at 12:22
`maplist=[['fromkey1','fromkey2','tokey1','tokey2']]; for mapping in maplist: newdict[mapping[0]][mapping[1]] = initial_dict[mapping[3]][mapping[4]]` obviously you will also have to look for the key2 being None and in those cases only use one level of dictionary access. — DisappointedByUnaccountableMod, Mar 09 '16 at 12:23
@gtlambert: why use `dict()`? Use a `{}` empty dictionary instead. — Martijn Pieters, Mar 09 '16 at 12:38
@MartijnPieters sure - I didn't actually realise that there is a difference between the two? Is there? Or is it just more readable to use `{}`? — gtlambert, Mar 09 '16 at 12:41
@gtlambert: see [Why is \[\] faster than list()?](https://stackoverflow.com/a/30216156). — Martijn Pieters, Mar 09 '16 at 12:42

eskaev · Accepted Answer · 2016-03-09T12:58:05.203

How about using dict.get? Instead of throwing an error, this returns None if the key isn't in the dictionary.

new_dict = {
    "content_text": initial_dict['content'].get('text'),
    "meta_title": initial_dict['meta'].get('title'),
    "meta_extras": {
        "time_related": {
            initial_dict['meta'].get('created')
        },
        "by": initial_dict['meta'].get('author')
    }
}

If this goes deeper than one level, you can do some_dict.get('key1', {}).get('key2') as was suggested in the comments.

Converting the original dict to a defaultdict is also an option, which allows you to keep using the [] notation (more practical than having to chain get methods):

from collections import defaultdict
def to_defaultdict(d):
    return defaultdict(lambda: None, ((k, to_defaultdict(v) if isinstance(v, dict) else v)
                                      for k, v in d.items()))
initial_dict = to_defaultdict(initial_dict)

You can then filter out the None values:

def filter_dict(d):
    return dict((k, filter_dict(v) if isinstance(v, dict) else v)
                for k, v in d.items() if v is not None)
new_dict = filter_dict(new_dict)

How to Pythonically map content from one dict to another in a fail safe manner?

1 Answers1