Removing mulitple key items with dict comprehension python

Question

I want to remove multiple key from my json and I'm using dictionary comprehensions like this

remove_key = ['name', 'description']

    data = {'full-workflow': {'task_defaults': {'retry': {'count': 3, 'delay': 2}}, 'tasks': {'t1': {'action': 'nn.postgres.export-db', 'description': 'Dump prod DB with default settings', 'input': {'database': 'prod', 'filepath': '/var/tmp/prod-dump.pgsql', 'host': 'postgres.local', 'password': 'mypass', 'username': 'myuser'}, 'name': 'db export', 'on-success': ['t2']}, 't2': {'action': 'nn.aws.upload-to-s3', 'description': 'Upload to S3 bucket for development', 'input': {'sourcepath': '{{ tasks(t1).result.filepath }}', 'targetpath': 's3://mybucket/prod-dump.pgsql'}, 'name': 'Upload to S3', 'on-success': ['t3'], 'retry': {'count': 5, 'delay': 5}}, 't3': {'action': 'nn.shell.command', 'description': 'Remove temp file from batch folder ', 'input': {'cmd': 'rm {{ tasks(t1).result.filepath }}'}, 'name': 'Remove temp file', 'on-complete': ['t4']}, 't4': {'action': 'nn.notify.send-mail', 'description': 'Send email to admin containing target path', 'input': {'from': 'bot@nn.io', 'message': 'DB Dump {{ tasks(t1).result.filepath }} was stored to S3', 'subject': 'Prod DB Backup', 'to': 'admin@nn.io'}, 'name': 'Send email', 'target': 'nn'}}}, 'version': '2'}

    def remove_additional_key(data):
        return {
            key: data[key] for key in data if key not in remove_key
        }

then just

new_data = remove_additional_key(data)

Because this is nested dict, I want to remove_key from tasks dict, so what am I doing wrong?

You have several dictionaries nested inside your main `data` dictionary. `name` and `description` are both keys in one of these internal dictionaries and that's why they aren't being removed. — yuvgin, Feb 03 '19 at 17:16
Your dictionary only has two keys that you are iterating over, `full-workflow` and `version`. — jordanm, Feb 03 '19 at 17:17
@PetarP if you have a clear idea of your desired action, you can edit your question. Do you want `name` and `description` to be removed if they are keys in any of the nested dictionaries? — yuvgin, Feb 03 '19 at 17:26

Rabbid76 · Accepted Answer · 2019-02-03T17:58:31.343

2

Your data are nested dictionaries. If you want to remove any data with a key contained in remove_key, then I suggest a recursive approach. This can be achieved, based on your exiting function remove_additional_key, with ease:

def remove_additional_key(data):

    sub_data = {} 
    for key in data:
        if key not in remove_key:
            if isinstance(data[key], dict): # if is dictionary
                sub_data[key] = remove_additional_key(data[key])
            else:
                sub_data[key] = data[key]
    return sub_data

new_data = remove_additional_key(data)

Note, if a entry is a dictionary can be checked by isinstance(data[key], dict). See How to check if a variable is a dictionary in Python?

edited Feb 03 '19 at 17:58

answered Feb 03 '19 at 17:30

Rabbid76

202,892
27
131
174

`isinstance` is fine, don't need to use `type` but this will delete the complete dict – copser Feb 03 '19 at 17:54
I tested it, and when I pass my dict in the function the output dict is empty – copser Feb 03 '19 at 17:57
yes, I've check, thanks for changing, this, you where faster :) – copser Feb 03 '19 at 17:59
@PetarP You're welcome. But note, you should consider the suggestions of the other answer, too. – Rabbid76 Feb 03 '19 at 18:02
1

I will, Alex suggestion and suggestion is noted, the hell of everything is that I need to do more cleaning, at least I have a nice base to play with – copser Feb 03 '19 at 18:06

score 1 · Answer 2 · answered Feb 03 '19 at 17:28

You have a dictionary with a few nested dictionaries. If you know exactly which subdictionary you have those keys to remove, you can use:

data['full-workflow']['tasks']['t1'].pop('name')

Using the lookup approach (key: data[key]) in a dictionary comprehension is inefficient however, on such a small data amount you won't notice a difference.

If you don't know the exact path to your nested dictionary, you can use a function (posting another answer for your convenience)

def delete_keys_from_dict(d, lst_keys):
    for k in lst_keys:
        try:
            del dict_del[k]
        except KeyError:
            pass
    for v in dict_del.values():
        if isinstance(v, dict):
            delete_keys_from_dict(v, lst_keys)

    return dict_del

Then you could call

delete_keys_from_dict(data, ['name', 'description'])

Needless to say, should you have name key in multiple nested dictionaries, all of them would be deleted, so be careful.

Removing mulitple key items with dict comprehension python

2 Answers2