Python list referencing same dictionary

Question

I encountered behavior in Python that I didn't expect when appending a dictionary to a list. Specifically, I tried

list_val=[]
local_dict={}
for val in ['a', 'b', 'c']:
    local_dict['first']=val
    list_val.append(local_dict)
    print(list_val)

which produces

[{'first': 'a'}]
[{'first': 'b'}, {'first': 'b'}]
[{'first': 'c'}, {'first': 'c'}, {'first': 'c'}]

Using the page pythontutor.com I'm able to watch the steps execute and I see that each list element points to the same dictionary that is being altered.

The fix is to declare the dictionary inside the loop,

list_val=[]
for val in ['a', 'b', 'c']:
    local_dict={}
    local_dict['first']=val
    list_val.append(local_dict)
    print(list_val)

which produces

[{'first': 'a'}]
[{'first': 'a'}, {'first': 'b'}]
[{'first': 'a'}, {'first': 'b'}, {'first': 'c'}]

The reason the behavior is unexpected is because I'm used to appending values to list like this:

list_val=[]
for val in ['a', 'b', 'c']:
    list_val.append(val)
    print(list_val)

which produces

['a']
['a', 'b']
['a', 'b', 'c']

Where should I expect to see pointer-like behavior?

I saw this question but my observation isn't related to functions.

Another relevant (but not quite duplicate) question: [List of lists changes reflected across sublists unexpectedly](https://stackoverflow.com/q/240178/364696) — ShadowRanger, Dec 14 '19 at 02:08
Does this answer your question? [List of lists changes reflected across sublists unexpectedly](https://stackoverflow.com/questions/240178/list-of-lists-changes-reflected-across-sublists-unexpectedly) — AMC, Dec 14 '19 at 03:03
The [lists of lists](https://stackoverflow.com/questions/240178/list-of-lists-changes-reflected-across-sublists-unexpectedly) is similar but less unexpected since the "* 3" operation is easy to understand as "make three copies of this variable". In my question the loop plays a similar role but I was more surprised that the list is pointing to the same variable. — Ben, Dec 14 '19 at 03:09
The two examples you've show work **exactly the same**. Everything in Python acts like a reference — juanpa.arrivillaga, Dec 14 '19 at 04:07

score 1 · Accepted Answer · answered Dec 14 '19 at 02:17

The pointer-like behavior is actually happening all the time. It's just that immutable types are safe; they always get replaced, even when you use operations (like +=) that logically mutate in place, but are not implemented that way for immutable objects.

You'll see this problem any time you repeatedly append the same object. In:

list_val=[]
local_dict={}
for val in ['a', 'b', 'c']:
    local_dict['first']=val
    list_val.append(local_dict)
    print(list_val)

you never rebind local_dict to a new object, so it's always the same dict, and you keep appending that same dict over and over. By contrast, in:

list_val=[]
for val in ['a', 'b', 'c']:
    local_dict={}
    local_dict['first']=val
    list_val.append(local_dict)
    print(list_val)

the line local_dict={} is rebinding the name local_dict to a completely new dict, unrelated to the one it used to be bound to.

As I mentioned, immutable types will behave "safely", so you can't just look at a portion of the code and know if it's safe without knowing the types involved. For example:

list_val = []
for val in ['a', 'b', 'c']:
    x += val
    list_val.append(x)
    print(list_val)

is entirely safe (inserting three unrelated str) if x is a str (immutable), but will insert three references to the same list if x is a list (mutable). If you're not sure, and need to make sure independent versions are inserted no matter what, you can copy (deep for safety, shallow for performance if you know only shallow copies are needed) at the moment of insertion, by importing the copy module, and replacing:

    list_val.append(x)

with:

    list_val.append(copy.deepcopy(x))

As it happens, for immutable types, copy.deepcopy is typically inexpensive (it pretends to copy, but doesn't actually do so, since it doesn't need to), so in cases where safety is paramount, it's a way of removing all doubt, without significantly reducing the performance when the types involved were already safe.

I think in your first paragraph your last line is meant to read "not implemented that way for immutable objects" ? — Ben, Dec 14 '19 at 03:12
@Ben: I'm not sure what you're getting at. That's exactly the wording I used... — ShadowRanger, Dec 14 '19 at 03:18

Piyush Singh · Answer 2 · 2019-12-14T02:33:57.583

You already noticed that each list_val.append call points to the same dictionary so when you update the dictionary, each item in your list gets updated. This is expected behaviour in python. This is because variables in python are all pointers to addresses (rather than references). This behavior is also manifest in other ways:

A =[1,2,3]
B=A
B.append(4)
print(A)
# prints [1,2,3,4] as opposed to [1,2,3]

This is because when I say B=A, they both point to the same address thereon { verify this: id(B)==id(A)}; so they'll always be same. Similarly in your case think that your list is made of pointers to whatever your local_dict defines (rather than the values), hence the behaviour. You can check out copy library for a workaround.

Python list referencing same dictionary

2 Answers2