How to merge two list of dictionaries based on a value

Question

I have two lists of dictionaries, lets say:

a = [{'id': 1, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}]

I want to have a list that merges every dictionary in both lists with the same ID. In this example i expect to have:

a = [{'id': 1, 'name': 'a', 'city': 'b'}]

Is there any cleaner way of doing it other than a for nested into the other?

Thanks

what should happen if two dictionaries have same keys (except the id key)? — Gilad Green, Sep 12 '20 at 18:27
Only one key per dictionary? Are these the best examples to show? Perhaps if you actually showed what you coded as a solution all would become clearer. — Booboo, Sep 12 '20 at 18:33

score 7 · Accepted Answer · answered Sep 12 '20 at 18:42

7

You can keep track of the ids with another dict (or defaultdict to make things simpler). Then update the items in that dict as you iterate. In the end the dict's values will have your list.

from collections import defaultdict
d = defaultdict(dict)

a = [{'id': 1, 'name': 'a'}, {'id': 3, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'city': 'c'}, {'id': 3, 'city': 'd'}]

for item in a + b:
    d[item['id']].update(item)
list(d.values())

# [{'id': 1, 'name': 'a', 'city': 'b'},
#  {'id': 3, 'name': 'a', 'city': 'd'},
#  {'id': 2, 'city': 'c'}]

Note this will overwrite duplicate values other than id — so if you have two with id: 1 and two different cities, you will only get the last city.

answered Sep 12 '20 at 18:42

Mark

90,562
7
108
148

1

Thats perfect. Thanks!! One more thing, if i want to add a default value, like 'NoName' if one of the keys doesnt exist, lets say, a = [{'id': 1, 'name': 'a'}, {'id': 3, 'name': 'a'}, {'id': 4, 'name': 'a'}] b = [{'id': 1, 'city': 'b'}, {'id': 2, 'city': 'c'}, {'id': 3, 'city': 'd'}] If i want the list to be: [{'id': 1, 'name': 'a', 'city': 'b'}, {'id': 3, 'name': 'a', 'city': 'd'}, {'id': 4, 'name': 'a', 'city': 'NoName'}, {'id': 2, 'city': 'c', 'name': 'NoName'}] Any pretty way other than checking in d if the key exists and if not, add it? – Javi DR Sep 12 '20 at 19:01
1

JaviDR - I’m out and typing on my phone so I can’t easily post code, but you could use something like defaultdict(lambda: {your defaults here}) to generate a populated dict as the default. – Mark Sep 12 '20 at 19:18

James Thompson · Answer 2 · 2020-09-12T18:44:38.983

1

One way to do this is to make a dictionary, mapping the identifier that you want to use (id in this case) to a dictionary of merged results.

#!/usr/bin/python

import collections

def merge_on_key(list_of_dictionaries, key, result):
    for d in list_of_dictionaries:
        assert(key in d)
        result[d[key]].update(d)

a = [{'id': 1, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'color': 'blue'}]

print 'a', a
print 'b', b

c = collections.defaultdict(lambda: {})
merge_on_key(a, 'id', c)
merge_on_key(b, 'id', c)

print 'merged results in dictionary with id 1', c[1]

That returns:

merged results in dictionary with id 1 {'city': 'b', 'id': 1, 'name': 'a'}

edited Sep 12 '20 at 18:44

answered Sep 12 '20 at 18:30

James Thompson

46,512
18
65
82

question states having lists of dictionaries and not just two dictionaries – Gilad Green Sep 12 '20 at 18:30
1

whoops! let me try and see if I can update sensibly – James Thompson Sep 12 '20 at 18:33

Anup Tiwari · Answer 3 · 2020-09-12T19:13:53.003

1

You can use map, lambda function in conjunction with update method for dictionaries, like this:

a = [{'id': 1, 'name': 'a'}, {'id': 2, 'name': 'a'}, {'id': 3, 'name': 'k'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'city': 'c'},  {'id': 4, 'city': 'cm'}]
a.extend(list(map(lambda x,y: y if x.get('id') != y.get('id') else x.update(y), a, b)))
a = list(filter(None, a))

a will now become a list containing dictionaries of merged values like this:

[{'id': 1, 'name': 'a', 'city': 'b'},
 {'id': 2, 'name': 'a', 'city': 'c'},
 {'id': 3, 'name': 'k'},
 {'id': 4, 'city': 'cm'}]

edited Sep 12 '20 at 19:13

answered Sep 12 '20 at 18:40

Anup Tiwari

474
2
5

For that to work you have to assume that `a[0].id == b[0].id`, which might not always be the case. – Pranav Hosangadi Sep 12 '20 at 18:43
Also, would you mind adding an explanation of what your code does? Remember that you are writing answers for a wide audience, and including _why_ this solution works makes your answer so much better. – Pranav Hosangadi Sep 12 '20 at 18:52
I also want it to add any item that is in b and not in a. For example: a = [{'id': 1, 'name': 'a'}] b = [{'id': 2, 'city': 'b'}, {'id': 1, 'city': 'c'}] Should have a = [{'id': 1, 'name': 'a', 'city': 'c'}, {'id': 2, 'city': 'b'} ] – Javi DR Sep 12 '20 at 18:54
Okay, I have added that to the solution. – Anup Tiwari Sep 12 '20 at 19:14

Soudipta Dutta · Answer 4 · 2022-12-13T12:21:27.697

from collections import defaultdict
from operator import itemgetter
l1 =[{'id': 1, 'City': 'Calcutta'}, {'id': 3, 'Country': 'Germany'}]
l2 = [{'id': 1, 'Country': 'India'}, {'id': 2, 'City': 'Delhi'}, {'id': 3, 'City': 'Berlin'}]

def merge1(l1,l2):
    d = defaultdict(dict)
    for l in (l1, l2):
        for innerdict1 in l:
            d[innerdict1['id']].update(innerdict1)

    l4 = sorted(d.values(), key=itemgetter("id"))
    l4p = print(l4)
    return l4p
merge1(l1, l2)

"""
[{'id': 1, 'City': 'Delhi', 'Country': 'India'}, {'id': 2, 'City': 'Calcutta'}, {'id': 3, 'Country': 'Germany', 'City': 'Berlin'}]

"""

How to merge two list of dictionaries based on a value

4 Answers4

Linked