python original list changes when try to modify a set of such lists

Question

I create two dictionaries: d1 and d2 and put them in a list c.

d1 = {'col1': [1, 2], 'col2': [3, 4]}
d2 = {'col1': [3, 6], 'col2': [5, 6]}
c=[d1,d2]

When I change an value in list c:

c[0]["col1"][0]=3

c
[{'col1': [3, 2], 'col2': [3, 4]}, {'col1': [3, 6], 'col2': [5, 6]}]

I surprisingly find that the specific value in original dictionary d1 also changed:

d1
{'col1': [3, 2], 'col2': [3, 4]}

Can anyone explain this to me? Why does d1 change together when I only try to modify values in list c?

So can I understand it this way that once I try to modify such a list, its original element (could be a dictionary, a list, or even a dataframe) will change at the same time?

You did not copy the dictionary, the list and variable refer to te *same* one, and hence the update is reflected through the other variable. — Willem Van Onsem, Feb 15 '18 at 10:01
See https://stackoverflow.com/questions/240178/list-of-lists-changes-reflected-across-sublists-unexpectedly for a similar scenario. — Willem Van Onsem, Feb 15 '18 at 10:02
You simply place d1 and d2 in c, but if you do c[0] you just access d1, and thus modify it. — Mathieu, Feb 15 '18 at 10:03

score 2 · Accepted Answer · answered Feb 15 '18 at 10:11

You asked python to insert your dictionary of lists into a list called c. This will not copy the internal contents of the lists in c (sometimes this behaviour is wanted). So c[0]['col1'][0] points to exactly the same float as d1['col1'][0]. Changing one also changes the other. If you want to create c which copies all the data in d1 and d2 do:

import copy
d1 = {'col1': [1, 2], 'col2': [3, 4]}
d2 = {'col1': [3, 6], 'col2': [5, 6]}
c = copy.deepcopy([d1,d2])

Thank you so much! Your answer is so much helpful! – D. Wei Feb 15 '18 at 17:27 — D. Wei, Feb 15 '18 at 17:27

score -1 · Answer 2 · answered Feb 15 '18 at 10:15

-1

This is entirely down to the mutability of lists - and how they are passed by reference.

You are not the first to be caught out by this, nearly every beginner will come across this at some point and get confused. The reason this is happening is that lists (and all other mutable data structures) are never copied in memory to different locations.

What I mean by this is when you call a function such as the following, you aren't actually passing in l, you are merely passing in what is essentially its location in memory - a reference to where it is.

def f(l):
    l[0] = 9

So that explains what may seem to be confusing behaviour of the following snippet:

>>> ls = [0, 1, 2]
>>> f(ls)
>>> ls
[9, 1, 2]

So to apply this to your scenario, when you create the c list, you are only storing references to the dictionaries (which are mutable in the same way lists are). Thus, when you modify either the reference in the list, or the dictionary itself, you are always modifying the same section in memory, so the change is reflected in the other variable.

To give one final simple example to demonstrate the mutable nature of dictionaries:

>>> d = {3 : 4}
>>> dd = d
>>> dd[3] = 5
>>> dd
{3: 5}
>>> d
{3: 5}

Finally, this post has a great explanation of Python variables if you are interested in further reading.

answered Feb 15 '18 at 10:15

Joe Iddon

20,101
7
33
54

There's no "pass by reference" (nor "pass by value" FWIW) in Python, and the statement that "when you call a function such as the following, you aren't actually passing in l, you are merely passing in what is essentially its location in memory - a reference to where it is." couldn't be more wrong - in Python when you pass an object as argument, you __really__ pass the object itself - no "reference" nor "location memory" nor nothing else that the object itself. – bruno desthuilliers Feb 15 '18 at 11:54
@brunodesthuilliers This was my own knowledge and understanding and I hadn't googled, but I just found [this](https://stackoverflow.com/a/986145/7434365), which seems to agree with me. What do you mean by you "really pass the object itself"? – Joe Iddon Feb 15 '18 at 12:16
Python's variables are not symbolic names for memory location, they are names bound to objects. So well yes, technically, at some point in the implementation (whether it's CPython, Jython, IronPython or else) there will be concepts like "memory locations", "references" (in the C++/Java acceptation) etc but semantically, at the python language level, you really just pass the object to the function, period. – bruno desthuilliers Feb 15 '18 at 13:48
@brunodesthuilliers Doesn't it just come down to semantics - my understanding is that objects have a single instance in memory and variables / parameters are just references to these. And it's only when explicit copies are made that a distinct memory instance is made. – Joe Iddon Feb 15 '18 at 16:13

python original list changes when try to modify a set of such lists

2 Answers2