Existing list overwritten when creating a new list

Question

In the script below, something strange happens when the second loop starts. The list area_1 changes value even though it is not involved in that loop at all. Same happens with area_2 after the third loop.

server_1 = 'x1'
server_2 = 'x2'
server_3 = 'x3'
table = [
    {'Terminal': 'site_1', 'City': 'London', 'Station': server_1},
    {'Terminal': 'site_2', 'City': 'London', 'Station': server_1},
    {'Terminal': 'site_3', 'City': 'London', 'Station': server_2},
    {'Terminal': 'site_4', 'City': 'New York', 'Station': server_3},
    {'Terminal': 'site_5', 'City': 'New York', 'Station': server_1},
    {'Terminal': 'site_6', 'City': 'New York','Station': server_2},
    {'Terminal': 'site_7', 'City': 'Tokyo', 'Station': server_3},
    {'Terminal': 'site_8', 'City': 'Tokyo', 'Station': server_3},
    {'Terminal': 'site_9', 'City': 'Tokyo', 'Station': server_2}    
]
city_index = [ 
    {'City':'London', 'Lat': 51.48,'Long': -0.14 },
    {'City':'New York', 'Lat': 40.69,'Long': -74.26 },
    {'City':'Tokyo', 'Lat': 35.51,'Long': 138.64 }
]
area_1 = []
for i in city_index:
    area_1.append(i)
    area_1[-1].update({'Station': server_1, 'Count of terminals': sum(q['City'] == i['City'] and q['Station'] == server_1 for q in table)})
area_2 = []
for i in city_index:
    area_2.append(i)
    area_2[-1].update({'Station': server_2, 'Count of terminals': sum(q['City'] == i['City'] and q['Station'] == server_2 for q in table)})
area_3 = []
for i in city_index:
    area_3.append(i)
    area_3[-1].update({'Station': server_3, 'Count of terminals': sum(q['City'] == i['City'] and q['Station'] == server_3 for q in table)})

all = area_1 + area_2 + area_3

for i in all:
    print(i)

Output: (Notice the station key)

{'City': 'London', 'Lat': 51.48, 'Long': -0.14, 'Station': 'x3', 'Count of terminals': 0}
{'City': 'New York', 'Lat': 40.69, 'Long': -74.26, 'Station': 'x3', 'Count of terminals': 1}
{'City': 'Tokyo', 'Lat': 35.51, 'Long': 138.64, 'Station': 'x3', 'Count of terminals': 2}
{'City': 'London', 'Lat': 51.48, 'Long': -0.14, 'Station': 'x3', 'Count of terminals': 0}
{'City': 'New York', 'Lat': 40.69, 'Long': -74.26, 'Station': 'x3', 'Count of terminals': 1}
{'City': 'Tokyo', 'Lat': 35.51, 'Long': 138.64, 'Station': 'x3', 'Count of terminals': 2}
{'City': 'London', 'Lat': 51.48, 'Long': -0.14, 'Station': 'x3', 'Count of terminals': 0}
{'City': 'New York', 'Lat': 40.69, 'Long': -74.26, 'Station': 'x3', 'Count of terminals': 1}
{'City': 'Tokyo', 'Lat': 35.51, 'Long': 138.64, 'Station': 'x3', 'Count of terminals': 2}

All 3 lists become identical at the end = area_3. Can someone please explain why this is happening? Why are area_1 and area_2 being overwritten?

Does this answer your question? [List of lists changes reflected across sublists unexpectedly](https://stackoverflow.com/questions/240178/list-of-lists-changes-reflected-across-sublists-unexpectedly) — Johan, Sep 19 '20 at 19:17

score 5 · Answer 1 · answered Sep 19 '20 at 19:21

5

Use copy module to copy objects istead of refering to them.

import copy
area.append(copy.copy(i))

Also, using i as iterator that is not an integer is very confusing, try to avoid it

answered Sep 19 '20 at 19:21

Alexander Golys

683
8
23

smac89 · Accepted Answer · 2020-09-19T20:00:02.197

1

area_1 and area_2 are not being overwritten. What's actually being overwritten is city_index. Try printing it after everything you do, and you will see.

To avoid this, you should copy the contents of city_index, instead of overwritting (dict.update(...)) them.

One way to do this is to find everywhere you do:

area_x.append(i)

Change it to

area_x.append(dict(**i)) # or area_x.append(dict(i))

This will force i to be copied rather than referenced.

edited Sep 19 '20 at 20:00

answered Sep 19 '20 at 19:18

smac89

39,374
15
132
179

@superbrain I guess it doesn't really matter. `dict(i)` works as well – smac89 Sep 19 '20 at 19:51
You are right, `city_index` is being modified, even though my code is not doing any changes on it, it just copies from it. Any idea why this is happening? – Skyoxic Sep 19 '20 at 19:53
@Skyoxic it's happening because of the `.update()` calls you make. `i` is an element of `city_index` and it is not copied when you add it to the list. So when you do `list[-1].update(...)`, you are actually updating `i` which is inside `city_index` – smac89 Sep 19 '20 at 19:58
thank you, but it's still not very clear to me. The function `.append()` supposedly copies a value and adds it to the list (I tested that in the interpreter by appending a variable to a list, then changing the value of that variable, the list remained unchanged). This was true for a string as an appended list element, but when I tried to reproduce the case with dictionaries, the function `.update()` seems to change the value of both the original dictionary and the appended list element. Was I not able to reproduce it with strings because they are immutable? – Skyoxic Sep 20 '20 at 09:19

Existing list overwritten when creating a new list

2 Answers2