Removing certain keys from a list of dictionary

Question

[{'YEAR': 2016.0, 'DOY': 310.0, 'temperature': 12.55, 'min_temp': 10.61, 'max_temp': 14.19, 'field_id': 1, 'date': Timestamp('2016-11-05 00:00:00')}, {'YEAR': 2016.0, 'DOY': 311.0, 'temperature': 14.89, 'min_temp': 12.6, 'max_temp': 17.49, 'field_id': 1, 'date': Timestamp('2016-11-06 00:00:00')}]

In the above list of dictionaries, I want to only retain the foll. keys in this list: ['YEAR', 'temperature']

From Filter dict to contain only certain keys?, I can filter the dict to only certain keys like this:

new_dict = { your_key: old_dict[your_key] for your_key in your_keys }

-- UPDATE: The soln should also work if a key from your_key does not exist in the dictionary

`[{ your_key: subdict[your_key] for your_key in your_keys } for subdict in dict_list ]` — juanpa.arrivillaga, Mar 26 '17 at 05:58
thanks @JohnGalt and @juanpa, your soln does not work if a key in `your_key` does not exist in the dictionary — user308827, Mar 26 '17 at 06:03
Quick fix: `[{ your_key: subdict[your_key] for your_key in your_keys if your_key in subdict} for subdict in dict_list ]` — juanpa.arrivillaga, Mar 26 '17 at 06:05

falsetru · Accepted Answer · 2017-03-26T06:08:27.053

Combine list comprehension with dict comprehension to get a list of dictionaries:

>>> lst = [{'YEAR': 2016.0, 'DOY': 310.0, 'temperature': 12.55,
            'min_temp': 10.61, 'max_temp': 14.19, 'field_id': 1,
            'date': Timestamp('2016-11-05 00:00:00')},
           {'YEAR': 2016.0, 'DOY': 311.0, 'temperature': 14.89,
            'min_temp': 12.6, 'max_temp': 17.49, 'field_id': 1,
            'date': Timestamp('2016-11-06 00:00:00')}]
>>> keys_to_retain = ['YEAR', 'temperature']
>>> [{key: d[key] for key in keys_to_retain} for d in lst]
[{'YEAR': 2016.0, 'temperature': 12.55}, {'YEAR': 2016.0, 'temperature': 14.89}]

UPDATE according to the question change

To make the dict comprehension work even if key from your_key does not exist in the dictionary, use if clause:

[{key: d[key] for key in keys_to_retain if key in d} for d in lst]
                                        ^^^^^^^^^^^

Alternatively, if you want the result dicitonaries have default value for missing key, you can use dict.get with default value instead of dict[..]:

[{key: d.get(key, 'default_value') for key in keys_to_retain} for d in lst]
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^

score 1 · Answer 2 · answered Mar 26 '17 at 06:42

One more solution using pop

mydict = [{'YEAR': 2016.0, 'DOY': 310.0, 'temperature': 12.55, 'min_temp': 10.61, 'max_temp': 14.19, 'field_id': 1}, 
          {'YEAR': 2016.0, 'DOY': 311.0, 'temperature': 14.89, 'min_temp': 12.6, 'max_temp': 17.49, 'field_id': 1}]

filter_list = ['YEAR','temperature']

# get all the keys to be removed. 
c =  list(set(mydict[0].keys())-set(filter_list))

# remove all the key,value for each dict in the list
[i.pop(cc) for i in mydict for cc in c]
print mydict

results in

[{'temperature': 12.55, 'YEAR': 2016.0}, {'temperature': 14.89, 'YEAR': 2016.0}]

Removing certain keys from a list of dictionary

2 Answers2