Updating elements in nested list using 'for' won't work in python list

Question

So I have a 4 by 3 nested list (4 rows, 3 columns) that I have nested as follows:

>>> c= [x[:] for x in [[None]*3]*4]
>>> print c
[[None, None, None], 
 [None, None, None], 
 [None, None, None], 
 [None, None, None]]

I have initialized my nested list in this fashion because this other SO question does a good job of explaining why some other methods don't work. (like c = [[None]*3]*4)

Now I want to update all elements in the first row to 0. i.e. I want to set all elements in

c[0] to 0. So I tried the following:
>>> for x in c[0]: x = 0
...
>>> c
[[None, None, None], [None, None, None], [None, None, None], [None, None, None]]
>>>

As you can see, the elements were not updated. The following worked however:

>>> c[0] = [0 for x in c[0]]
>>>
>>> c
[[0, 0, 0], [None, None, None], [None, None, None], [None, None, None]]

And I was almost sure it would because I'm creating a new list of 0s and assigning it to c[0].

Anyway, I then went on to use the for loop and tried to update the first column (i.e. the first element of every row) to 0 and that worked.

>>> for x in c: x[0] = 0
...
>>> c
[[0, None, None], [0, None, None], [0, None, None], [0, None, None]]

I understand that this for loop update is different from the previous for loop update since the first one tried to loop over single elements while this one loops over lists and just accesses the first element of each list.

I'm sure I'm missing something about names pointing to other names but I can't put my finger on what the exact issue is here. Could someone please help?

Answer 1

for x in c[0]: x = 0

In this loop, you're actually creating new references to integers present in that list and then changing those new references.

As integers are immutable, so the original reference are not going to be affected. Plus, as assignment is not an in-place operation, so this won't affect mutable objects as well.

>>> a = b = 1
>>> b += 1     # in-place operation, this will work differently for mutable objects
>>> a, b       # a is still unchanged
(1, 2)

Assignments operations won't affect mutable objects as well:

>>> x = y = [1, 1]
>>> x = 2 # `x` now points to a new object, number of references to [1, 1] decreased by 1
>>> x, y
(2, [1, 1])

But in-place operations will:

>>> x = y = [1, 1]
>>> x.append(2)
>>> x, y
([1, 1, 2], [1, 1, 2])

So, that above loop is actually equivalent to:

x = c[0]
x = 0    #this won't affect `c[0]`
x = c[1]
x = 0
...

---

In this loop you actually changed c[0] to point to a new list object:

>>> c= [x[:] for x in [[None]*3]*4]
>>> c= [x[:] for x in [[None]*3]*4]
>>> print id(c[0])
45488072
>>> c[0] = [0 for x in c[0]]
>>> print id(c[0])              #c[0] is a new list object
45495944

Answer 2

In each case, x is a reference to something.

In the case where it is a reference to None, you're creating an instance of the int(0) and switching x over to reference that. So here c is not involved at all.

In the other cases x is a reference to a component of c, so when you modify that component, you see the change reflected in c

Answer 3

Now I want to update all elements in the first row to 0. i.e. I want to set all elements in c[0] to 0. So I tried the following:

>>> for x in c[0]: x = 0

>>> c
[[None, None, None], [None, None, None], [None, None, None], [None, None, None]]
>>>

As you can see, the elements were not updated.

That's because x was given each of the None values in c[0] in turn, so you're effectively saying:

x = None
x = 0
x = None
x = 0
x = None
x = 0

This does nothing useful.

The following worked however:

>>> c[0] = [0 for x in c[0]]
>>>
>>> c
[[0, 0, 0], [None, None, None], [None, None, None], [None, None, None]]

And I was almost sure it would because I'm creating a new list of 0s and assigning it to c[0].

This works because c[0] is a reference to, or alias for, the first element (list) in c... you can assign a new value into it and it affects the original c container. The crucial thing here is the write-through reference / alias idea. I'm not sure what the python community tends to call this (I'm mainly a C++ programmer), but functionally your c[0] refers to that element in c and allows you to overwrite it. c[0] isn't like x above, as x effectively saw an immutable copy of the element values (None), but that value maintained no connection to or ability to write back into the container c. That's because None was a simple "scalar" value.

This idea that variables sometimes effectively reference immutable copies of values assigned to them that further assignment will disassociate them with, and other times actually continue to refer to the assigned value itself in such a way that a further write lets it be modified, is actually common to several interpreted languages (e.g. Ruby does it too). It's confusing at first! These languages want you to think they're simpler to use than say C++ where these "references" are explicit and have their own syntax/notation, but in reality you soon get bitten and have to learn to understand the difference anyway, then start using copy.deepcopy when you need a real independent copy of data. They want to seem intuitive, but the intuitive thing - deep copying values - is too expensive to do for large containers and objects: most newbie's programs would get horrifically slow and they wouldn't know why. So, modern scripting languages tend to instead adopt this dangerous behaviour of letting you write to things that you probably thought you'd safely copied some data out of. After a couple weeks of using the language it probably all clicks.... While initially confusing, it is actually useful too - having to refer to the data using the original variable name, all the keys and indices to narrow things down to a specific scalar value can be painfully verbose, and of course this is a necessary abstraction to allow functions to operate on parts of complex objects/containers they're passed without having to know about the rest of the object....

Anyway, I then went on to use the for loop and tried to update the first column (i.e. the first element of every row) to 0 and that worked.

>>> for x in c: x[0] = 0
...
>>> c
[[0, None, None], [0, None, None], [0, None, None], [0, None, None]]

This works because x is bound as a reference/alias to each list in turn, and as lists are "complex" objects the reference/alias allows that write-through capability I described above.

I understand that this for loop update is different from the previous for loop update since the first one tried to loop over single elements while this one loops over lists and just accesses the first element of each list.