iteratively appending N items to list gives last item N times instead

Question

I am accessing a list of dictionary items list_of_dict = [{'ka':'1a', 'kb':'1b', 'kc':'1c'},{'ka':'2a'},{'ka':'3a', 'kb':'3b', 'kc':'3c'}], and trying to conditionally append each entry to another list article_list using a function add_entries.

My desired output

article_list = [{x:1a, y:1b, z:1c}, {x:2a}, {x:3a, y:3b, z:3c}]

My code:


def add_entries(list_of_dict):
   keys = ['x','y','z']
   #defining a temporary dictionary here
   my_dict = dict.fromkeys(keys,0)
   entry_keys = ['ka','kb','kc']

   my_list = []

   for item in list_of_dict:
    # conditionally append the entries into the temporary dictionary maintaining desired key names
        my_dict.update({a: item[b] for a,b in zip(keys, entry_keys) if b in item})
        my_list.append(my_dict)

   return my_list



if __name__ == "__main__":
   list_of_dict = [{'ka':'1a', 'kb':'1b', 'kc':'1c'},{'ka':'2a'},{'ka':'3a', 'kb':'3b', 'kc':'3c'}]
   article_list = []
   returned_list = add_entries(list_of_dict)
   article_list.extend(returned_list)

My output

article_list = [{x:3a, y:3b, z:3c}, {x:3a, y:3b, z:3c}, {x:3a, y:3b, z:3c}]

L.Grozinger · Accepted Answer · 2020-02-07T17:54:38.863

Whats wrong

my_list.append(my_dict) appends a reference to the my_dict object to my_list. Therefore, at the end of the for loop in your example, my_list is a list of references to the exact same dictionary in memory.

You can see this for yourself using the function id. id, at least in CPython, basically gives you the memory address of an object. If you do

article_list.extend(returned_list)
print([id(d) for d in article_list])

You'll get a list of identical memory addresses. On my machine I get

[139920570625792, 139920570625792, 139920570625792]

The consequence is that updating the dictionary affects 'all of the dictionaries' in your list. (quotes because really there are not multiple dictionaries in your list, just many times the exact same one). So in the end, only the last update operation is visible to you.

A good discussion on references and objects in Python can be found in this answer https://stackoverflow.com/a/30340596/8791491

The fix

Moving the definition of my_dict inside the for loop, means that you get a new, separate dictionary for each element of my_list. Now, the update operation won't affect the other dictionaries in the list, and my_list is a list of references to several different dictionaries.

def add_entries(list_of_dict):
    keys = ['x','y','z']
    entry_keys = ['ka','kb','kc']

    my_list = []

    for item in list_of_dict:
        #defining a new dictionary here
        my_dict = dict.fromkeys(keys,0)

        # conditionally append the entries into the temporary dictionary maintaining desired key names
        my_dict.update({a: item[b] for a,b in zip(keys, entry_keys) if b in item})
        my_list.append(my_dict)

    return my_list

score 1 · Answer 2 · answered Feb 07 '20 at 17:14

1

You can use this.

keys=['x','y','z']
res=[{k1:d[k]} for d in list_of_dict for k,k1 in zip(d,keys)]

output

[{'x': '1a'},
 {'y': '1b'},
 {'z': '1c'},
 {'x': '2a'},
 {'x': '3a'},
 {'y': '3b'},
 {'z': '3c'}]

answered Feb 07 '20 at 17:14

Ch3steR

20,090
4
28
58

score 0 · Answer 3 · answered Feb 07 '20 at 17:15

0

Try this:

list_new_d=[]
for d in list_of_dict:
  new_d={}

  for k,v in d.items():
    if k == 'ka':
      new_d['x'] = v

    if k == 'kb':
      new_d['y'] = v

    if k == 'kc':
      new_d['z'] = v
  list_new_d.append(new_d)

answered Feb 07 '20 at 17:15

Shubham Shaswat

1,250
9
14

iteratively appending N items to list gives last item N times instead

3 Answers3