python lookup between dictionaries

Question

Is there a way to speed this up? currently using 2 for loops (considering duplicates). any suggestions on speeding this up?

d1 = [{'key': 't1', 'val': 1}, {'key': 't2', 'val': 2}, {'key': 't3', 'val': 3}, {'key': 't4', 'val': 4}, {'key': 't5', 'val': 5}, {'key': 't6', 'val': 6}, {'key': 't7', 'val': 7}, {'key': 't8', 'val': 8}, {'key': 't9', 'val': 9}, {'key': 't10', 'val': 10}, {'key': 't11', 'val': 11}, {'key': 't12', 'val': 12}, {'key': 't13', 'val': 13}, {'key': 't14', 'val': 14}, {'key': 't15', 'val': 15}, {'key': 't16', 'val': 16}, {'key': 't17', 'val': 17}, {'key': 't18', 'val': 18}, {'key': 't19', 'val': 19}]

d2 = [{'key': 't1', 'val': 'newval1'}, {'key': 't1', 'val': 'newval11'}, {'key': 't2', 'val': 'newval2'}, {'key': 't3', 'val': 'newval3'}, {'key': 't6', 'val': 'newval6'}, {'key': 't7', 'val': 'newval7'}, {'key': 't8', 'val': 'newval8'}, {'key': 't9', 'val': 'newval9'}, {'key': 't10', 'val': 'newval10'}, {'key': 't11', 'val': 'newval11'}, {'key': 't12', 'val': 'newval12'}, {'key': 't13', 'val': 'newval13'}, {'key': 't14', 'val': 'newval14'}, {'key': 't15', 'val': 'newval15'}, {'key': 't16', 'val': 'newval16'}, {'key': 't17', 'val': 'newval17'}, {'key': 't18', 'val': 'newval18'}, {'key': 't19', 'val': 'newval19'}]


>>> for x in d1:
...     for y in d2:
...         if x['key'] == y['key']:
...             print(x['key'], x['val'], y['val'])
...
('t1', 1, 'newval1')
('t1', 1, 'newval11')
('t2', 2, 'newval2')
('t3', 3, 'newval3')
('t6', 6, 'newval6')
('t7', 7, 'newval7')
('t8', 8, 'newval8')
('t9', 9, 'newval9')
('t10', 10, 'newval10')
('t11', 11, 'newval11')
('t12', 12, 'newval12')
('t13', 13, 'newval13')
('t14', 14, 'newval14')
('t15', 15, 'newval15')
('t16', 16, 'newval16')
('t17', 17, 'newval17')
('t18', 18, 'newval18')
('t19', 19, 'newval19')

Does this answer your question? [Most Pythonic way to merge two dictionnaries having common key/value pair](https://stackoverflow.com/questions/71649085/most-pythonic-way-to-merge-two-dictionnaries-having-common-key-value-pair) — matszwecja, Mar 30 '22 at 08:05

S.B · Accepted Answer · 2022-03-30T08:27:53.923

With this structure No... They are two lists, you need to iterate over them to find what you are looking for.

But you can store this data as a dictionary of dictionaries using their keys: (After confirmation from you that d1 always have unique keys, just turn d1 to a dictionary of dictionaries)

d1 = {d['key']: {'val': d['val']} for d in d1}

This way you can iterate over d2(a single for loop) and pick the relevant value from d1.

for d in d2:
    key, value = d['key'], d['val']
    print(key, d1[key]['val'], value)

here is the full code:

d1 = [{'key': 't1', 'val': 1}, {'key': 't2', 'val': 2}, {'key': 't3', 'val': 3},
      {'key': 't4', 'val': 4}, {'key': 't5', 'val': 5}, {'key': 't6', 'val': 6},
      {'key': 't7', 'val': 7}, {'key': 't8', 'val': 8}, {'key': 't9', 'val': 9},
      {'key': 't10', 'val': 10}, {'key': 't11', 'val': 11},
      {'key': 't12', 'val': 12}, {'key': 't13', 'val': 13},
      {'key': 't14', 'val': 14}, {'key': 't15', 'val': 15},
      {'key': 't16', 'val': 16}, {'key': 't17', 'val': 17},
      {'key': 't18', 'val': 18}, {'key': 't19', 'val': 19}]

d2 = [{'key': 't1', 'val': 'newval1'}, {'key': 't1', 'val': 'newval11'},
      {'key': 't2', 'val': 'newval2'}, {'key': 't3', 'val': 'newval3'},
      {'key': 't6', 'val': 'newval6'}, {'key': 't7', 'val': 'newval7'},
      {'key': 't8', 'val': 'newval8'}, {'key': 't9', 'val': 'newval9'},
      {'key': 't10', 'val': 'newval10'}, {'key': 't11', 'val': 'newval11'},
      {'key': 't12', 'val': 'newval12'}, {'key': 't13', 'val': 'newval13'},
      {'key': 't14', 'val': 'newval14'}, {'key': 't15', 'val': 'newval15'},
      {'key': 't16', 'val': 'newval16'}, {'key': 't17', 'val': 'newval17'},
      {'key': 't18', 'val': 'newval18'}, {'key': 't19', 'val': 'newval19'}]

d1 = {d['key']: {'val': d['val']} for d in d1}

for d in d2:
    key, value = d['key'], d['val']
    print(key, d1[key]['val'], value)

output:

t1 1 newval1
t1 1 newval11
t2 2 newval2
t3 3 newval3
t6 6 newval6
t7 7 newval7
t8 8 newval8
t9 9 newval9
t10 10 newval10
t11 11 newval11
t12 12 newval12
t13 13 newval13
t14 14 newval14
t15 15 newval15
t16 16 newval16
t17 17 newval17
t18 18 newval18
t19 19 newval19

Thanks! please note that d2 has duplicates. with what you suggested, the result would be wrong. >>> d2 = [{'key': 't1', 'val': 'newval1'}, {'key': 't1', 'val': 'newval11'}] >>> d2 = {d['key']: {'val': d['val']} for d in d2} >>> d2 {'t1': {'val': 'newval11'}} — user3327034, Mar 30 '22 at 08:10
@user3327034 Oh right. What about `d1` ? Does it always have unique keys? — S.B, Mar 30 '22 at 08:15
@user3327034 So that can be fixed, `d2` can still be a list of dictionaries. I will re-write my answer in a second — S.B, Mar 30 '22 at 08:18
thanks for the update. How is this better than 2 for loops that I have? — user3327034, Mar 30 '22 at 08:24
@user3327034 Time complexity of two nested loops is O(n^2) but a single loop is O(n). So it's much faster if your data is big enough. — S.B, Mar 30 '22 at 08:26
@user3327034 I depends on your need. If you're going to retrieve data several times, then it worth it. Because the creation of that dict is once. you pay once for it then you will get much more performance for next iterations. (considering long-running applications) — S.B, Mar 30 '22 at 08:36

python lookup between dictionaries

1 Answers1