append all values in a dict if all elements exists and remove duplicates

Question

I have a scenario where I have three dictionaries which I want merge into one but the condition is while I compare the three dictionaries with key name if there are duplicates need to remove them.

Here is what I have tried :

dict1= {'d1': [{'name': 'app1', 'id': 7134}, {'name': 'app2', 'id': 242}, {'name': 'yest app', 'id': 67},{'name': 'abc jam app', 'id': 6098}]}

dict2= {'d2': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}]}

dict3= {'d3': [{'name': 'app1', 'id': 70}, {'name': 'app2', 'id': 2582},{'name': 'availabla2z', 'id': 6667}]}

dict2 = {i:j for i,j in dict2.items() if i not in dict1}
dict3 = {i:j for i,j in dict3.items() if i not in dict2}

But the same do not give results also I am not sure how to compare three dicts for that matter. and since if you look at the data dict1 is having an element 'name': 'app1' where as the same element is there in dict2 like this 'name': 'app1 ' (with a space) not sure how to format this as well and get a final dict like below as result.

{'final': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17},{'name': 'abc jam app', 'id': 6098},{'name': 'availabla2z', 'id': 6667}]}

@snakecharmerb Python 3 would be fine, is there any major difference in use od modules in them if yes, then I would like to know both. — John, Apr 16 '20 at 07:09
How do you select which dict to keep. Ex `{'name': 'app1 ', 'id': 30}` and not `{'name': 'app1', 'id': 7134}` or `{'name': 'app1', 'id': 70}`? — Rakesh, Apr 16 '20 at 07:22
@Rakesh I would keep any one of them but not with the space one so can `strip()` be used later if there are trailing whitespaces in the final dict name ? — John, Apr 16 '20 at 07:45

arnaud · Answer 1 · 2020-04-16T07:52:00.107

1

Here's a solution, taking advantage of this other SO answer (useful for python-2.x alternatives) that will remove duplicates without any particular rule:

final_dict = dict()
final_dict["final"] = dict1["d1"] + dict2["d2"] + dict3["d3"]
final_dict["final"] = list(
    {v['name'].strip():v for v in final_dict["final"]}.values()
)  # see usage of .strip() to handle space problems you mention

print(final_dict)

Result:

{'final': [
  {'name': 'app1', 'id': 70},
  {'name': 'app2', 'id': 2582},
  {'name': 'yest app', 'id': 17},
  {'name': 'abc jam app', 'id': 6098},
  {'name': 'availabla2z', 'id': 6667}]
}

edited Apr 16 '20 at 07:52

answered Apr 16 '20 at 07:28

arnaud

3,293
1
10
27

there is no such restriction in that like which name to keep from which dict if it matches in all 3 dicts then keep any one of them in the final dict – John Apr 16 '20 at 07:42
Then the answer I suggested works: you'd keep any of them. – arnaud Apr 16 '20 at 07:49

score 0 · Answer 2 · answered Apr 16 '20 at 07:22

Here is a working Updater code:

dict1= {'d1': [{'name': 'app1', 'id': 7134}, {'name': 'app2', 'id': 242}, {'name': 'yest app', 'id': 67},{'name': 'abc jam app', 'id': 6098}]}
dict2= {'d2': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}]}
dict3= {'d3': [{'name': 'app1', 'id': 70}, {'name': 'app2', 'id': 2582},{'name': 'availabla2z', 'id': 6667}]}
final = {'final':[]}
for i in dict1['d1']:
    final['final'].append(i)

for k,l in zip(dict3['d3'],range(len(dict1['d1']))):
    if k['name'] == final['final'][l]['name']:
        final['final'][l].update(k)
    else:
        final['final'].append(k)
for j,l in zip(dict2['d2'],range(len(dict1['d1']))):
    if j['name'].strip() == final['final'][l]['name'].strip():
        final['final'][l].update(j)
    else:
        final['final'].append(j)

This gives:

{'final': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}, {'name': 'abc jam app', 'id': 6098}, {'name': 'availabla2z', 'id': 6667}]}

RoadRunner · Answer 3 · 2020-04-16T07:52:29.480

You could group all dictionaries together by name using defaultdict:

from collections import defaultdict

d = defaultdict(list)
for lst in (dict1.values(), dict2.values(), dict3.values()):
    for sublst in lst:
        for dic in sublst:
            d[dic["name"].strip()].append(dic)

Then choose the dictionaries with the smallest id value using min(). This still works for the requirements since it still chooses one dictionary and matches the output requested.

from operator import itemgetter

result = {'field': [min(x, key=itemgetter('id')) for x in d.values()]}

print(result)

Output:

{'field': [{'name': 'app1', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}, {'name': 'abc jam app', 'id': 6098}, {'name': 'availabla2z', 'id': 6667}]}

append all values in a dict if all elements exists and remove duplicates

3 Answers3