Merge two list of dicts in python when some dicts are duplicates

Question

I have two list of dictionaries as follows:

x=[{'id':1, 'var1':2},
   {'id':1, 'var1':2},
   {'id':2, 'var1':2}]

y=[{'var2':1, 'var3':2},
   {'var2':3, 'var3':3},
   {'var2':4, 'var3':4}]

I want to merge each dict into each other dict, so the output is as follows:

z=[{'id':1, 'var1':2, 'var2':1, 'var3':2},
   {'id':1, 'var1':2, 'var2':3, 'var3':3},
   {'id':2, 'var1':2, 'var2':4, 'var3':4}]

However, when I use the following merge function:

def merge_x_and_y(x_data, y_data):
    for x_entry, y_entry in zip(x_data, y_data):
        x_entry.update(y_entry )
    return x_data

I instead end up with:

z=[{'id':1, 'var1':2, 'var2':1, 'var3':2},
   {'id':1, 'var1':2, 'var2':1, 'var3':2},
   {'id':2, 'var1':2, 'var2':2, 'var3':3}]

So if a dict in x is a duplicate, it duplicates the dict from y, which I don't want. I want each x dict to be merged with a different y dict (even if the x dict is a duplicate).

Is there a way to merge each x dict with a y dict ignoring duplicates?

Edit:

The reprex above works, but I still don't get the right outcome in my actual code. I think it might be to do with Lenik's suggestion, that the same elements may be referenced twice.

Prior to the merge, I've tried to expand the data using:

def expand_x(x):
    processed_x_data = []     
    [processed_x_data .extend([entry]*entry['count']) for entry in x]
    return processed_x_data

After doing this step, should I reallocate a unique id to each entry to avoid double referencing?

This isn't how dictionaries are made in python. They should instead look like {key1: value1, key2: value2} — Brian, Jul 29 '18 at 06:49
there are vary method to merge dict, but the dict is build in type. https://stackoverflow.com/questions/38987/how-to-merge-two-dictionaries-in-a-single-expression — Charles Cao, Jul 29 '18 at 06:56
i second @RafaelC. When i use your code, (i changed your variables to strings) and `z = merge_x_y(x, y)` i get the expected output. It is worth noting that `z==x` so that if you do something to `x` afterwards, that will affect `z` — e.s., Jul 29 '18 at 07:10

lenik · Accepted Answer · 2018-07-29T07:42:48.200

2

I have a strong impression, that duplicate elements in the first dictionary are not two different elements, but the same element, referenced twice. Try this code to merge dictionaries:

def merge_x_and_y(x_data, y_data):
    result = [a.copy() for a in x_data]
    [a.update(b) for a, b in zip(result, y_data)]
    return result

and tell me if it does the trick =)

edited Jul 29 '18 at 07:42

answered Jul 29 '18 at 07:01

lenik

23,228
4
34
43

Hi Lenik, I've expanded the question. Thanks, Ed – Thirst for Knowledge Jul 29 '18 at 07:30
Your suggestion returned 'None' for all entries when i tried it. – Thirst for Knowledge Jul 29 '18 at 07:31
@ThirstforKnowledge sorry, hopefully updated answer fixes the problem – lenik Jul 29 '18 at 07:43

score 0 · Answer 2 · answered Jul 29 '18 at 07:12

I have to make an assumption about the keys in the dictionaries, but try this :

id = "id"
var1 = "v1"
var2 = "v2"
var3 = "v3"

x=[{id:1, var1:2},
   {id:1, var1:2},
   {id:2, var1:2}]

y=[{var2:1, var3:2},
   {var2:3, var3:3},
   {var2:4, var3:4}]

z = []
for i in range(len(x)) :
    z.append(dict(**x[i],**y[i]))

for i in z :
    print(i)

Merge two list of dicts in python when some dicts are duplicates

2 Answers2