Nested List Indices

Question

I have experienced some problem by using a nested list in Python in the code shown bleow.

Basically, I have a 2D list contains all 0 values, I want to update the list value in a loop.

However, Python does not produce the result I want. Is there something that I misunderstand about range() and Python list indices?

some_list = 4 * [(4 * [0])]
for i in range(3):
    for j in range(3):
        some_list[i+1][j+1] = 1
for i in range(4):
    print(some_list[i])

The results I expected are:

[0, 0, 0, 0]
[0, 1, 1, 1]
[0, 1, 1, 1]
[0, 1, 1, 1]

But the actual results from Python are:

[0, 1, 1, 1]
[0, 1, 1, 1]
[0, 1, 1, 1]
[0, 1, 1, 1]

What's going on here?

Here is a link to a guide on programming idiomatic Python. Some of it is outdated, but the part about variables and names continues to be relevant: http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#other-languages-have-variables — pcurry, Jun 11 '13 at 21:27

score 22 · Accepted Answer · answered Oct 24 '12 at 22:28

22

The problem is caused by the fact that python chooses to pass lists around by reference.

Normally variables are passed "by value", so they operate independently:

>>> a = 1
>>> b = a
>>> a = 2
>>> print b
1

But since lists might get pretty large, rather than shifting the whole list around memory, Python chooses to just use a reference ('pointer' in C terms). If you assign one to another variable, you assign just the reference to it. This means that you can have two variables pointing to the same list in memory:

>>> a = [1]
>>> b = a
>>> a[0] = 2
>>> print b
[2]

So, in your first line of code you have 4 * [0]. Now [0] is a pointer to the value 0 in memory, and when you multiply it, you get four pointers to the same place in memory. BUT when you change one of the values then Python knows that the pointer needs to change to point to the new value:

>>> a = 4 * [0]
>>> a
[0, 0, 0, 0]
>>> [id(v) for v in a]
[33302480, 33302480, 33302480, 33302480]
>>> a[0] = 1
>>> a
[1, 0, 0, 0]

The problem comes when you multiply this list - you get four copies of the list pointer. Now when you change one of the values in one list, all four change together:

>>> a[0][0] = 1
>>> a
[[1, 0, 0, 0], [1, 0, 0, 0], [1, 0, 0, 0], [1, 0, 0, 0]]

The solution is to avoid the second multiplication. A loop does the job:

>>> some_list = [(4 * [0]) for _ in range(4)]

answered Oct 24 '12 at 22:28

D Read

3,175
1
15
25

2

Thanks for the in-depth explanation! – Ken Ma Oct 24 '12 at 22:48
6

While the code here solves the OP's problem, and the explanation correctly identifies that this is down to the lists being passed around 'by pointer', I still feel compelled to downvote just because of the first two lines. Variables in Python are not 'normally passed by value'. Everything in Python is passed 'by pointer', but ints and strings (for example) are immutable so Python's behavior with regards to those types can be considered *effectively equivalent* to passing by value for almost all real world purposes. – Mark Amery Dec 04 '12 at 20:46
The 'almost' in the above comment is, of course, because you don't significantly increase your memory usage by assigning the same enormous string to multiple variables or passing it into functions as a parameter - which you would if the enormous string were passed by value. – Mark Amery Dec 04 '12 at 20:49
2

Also, it's worth noting that if you replace the `int` literals `1` and `2` in the first code block with `list` literals `[1]` and `[2]`, the behavior remains the same (i.e. '[1]' is printed, not '[2]'). Suggesting that this is a type-related issue implies that you would get different behavior if you replaced the `int` literals in the code block with `list` literals, which is untrue and potentially very misleading for a Python newbie. – Mark Amery Dec 04 '12 at 20:53
3

1. Since ints and strings are 'effectively' passed by value, then that explanation is a useful level of abstraction to describe the behaviour to the OP. 2. Most newbies deal with ints and strings most of the time and are basic building blocks, suggesting my use of 'normally' isn't so far wrong. I think your comments are useful and interesting details. BUT I think that voting this helpful answer down to only 2 is constructive behaviour, when there's nothing actually wrong about it and the OP is so pleased with the explanation. Whereas the other answer has few words, a serious flaw and scores 7. – D Read Dec 05 '12 at 00:37
thanks for your time and explanation. it was pretty good. – James Sapam Dec 05 '13 at 16:46

Ashwini Chaudhary · Answer 2 · 2014-02-13T19:57:53.227

Actually all the objects in your list are same, so changing one changes others too:

In [151]: some_list = 4 * [(4 * [0])]  

In [152]: [id(x) for x in some_list]
Out[152]: [148641452, 148641452, 148641452, 148641452]

In [160]: some_list[0][1]=5  #you think you changed the list at index 0 here

In [161]: some_list
Out[161]: [[0, 5, 0, 0], [0, 5, 0, 0], [0, 5, 0, 0], [0, 5, 0, 0]]  #but all lists are changed

Create your list this way:

In [156]: some_list=[[0]*4 for _ in range(4)]

In [157]: some_list
Out[157]: [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]

In [158]: [id(x) for x in some_list]
Out[158]: [148255436, 148695180, 148258380, 148255852]

In [163]: some_list[0][1]=5

In [164]: some_list
Out[164]: [[0, 5, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]  #works fine in this case

You make an unnecessary expansion to [0 for _ in range(4)] - you can just use 4 * [0]. See my explanation in the other post. — D Read, Oct 25 '12 at 09:17

Nested List Indices

2 Answers2

Linked

Related