Explanation of python reference behavior when creating list of lists

Question

When creating a new nested list the following way:

test_list = [[""]]*5

unexpected behavior happens, appending to an index the following modifications happens:

test_list[1].append(2)
[['', 2], ['', 2], ['', 2], ['', 2], ['', 2]]

when doing the following it works as I would expect:

test_list[1] = test_list[1] + [0]
[[''], ['', 0], [''], [''], ['']]

but when using the += operator the same odd thing happens

test_list[1] += [0]
[['', 0], ['', 0], ['', 0], ['', 0], ['', 0]]

when the list is defined as following: [[""] for i in range(len(5))] all the examples return the expected output.

What is going on? Is it some reference thing I am not understanding?

That's what I don't like most in languages like Java, Python, etc - They're using references implicitly. Of course, once you know that, you can handle it. But C++ makes it obvious - which I first didn't like, but now really appreciate. — Tobias Brösamle, Mar 20 '19 at 14:52
Be careful of this *gotcha* too: [“Least Astonishment” and the Mutable Default Argument](https://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument) — wwii, Mar 20 '19 at 15:00

Yuan JI · Answer 1 · 2019-06-05T09:37:52.970

By using test_list = [[""]] * 5 you are creating 5 references to the same list. This means there is ONLY ONE ROW in the memory.

When you modify a row, if affects the row in memory, and since all other "rows" refer to it, they will simply use the list in memory which is up-to-date.

This however creats a list of 5 lists in memory:

test_list = [[""] for i in range(len(5))]

You could find an example and explanation in wtfpython:

# Let's initialize a row
row = [""]*3 #row i['', '', '']
# Let's make a board
board = [row]*3

Output:

>>> board
[['', '', ''], ['', '', ''], ['', '', '']]
>>> board[0]
['', '', '']
>>> board[0][0]
''
>>> board[0][0] = "X"
>>> board
[['X', '', ''], ['X', '', ''], ['X', '', '']]

Explanation:

When we initialize row variable, this visualization explains what happens in the memory.

And when the board is initialized by multiplying the row, this is what happens inside the memory (each of the elements board[0], board[1] and board[2] is a reference to the same list referred by row)

We can avoid this scenario here by not using row variable to generate board. (Asked in this issue).

>>> board = [['']*3 for _ in range(3)]
>>> board[0][0] = "X"
>>> board
[['X', '', ''], ['', '', ''], ['', '', '']]

Shahaf Shavit · Accepted Answer · 2019-03-20T15:26:59.107

You are creating a list that has five elements which point to the same list.

When you call test_list[1].append(2), it appends number 2 to the list, which appends it to all lists.

When you call test_list[1] = test_list[1] + [0], a new instance of a list is created, and is assigned to test_list[1]. Now in index 1 you have a different list than all other 4.

Here, test_list[1] += [0], += is translated to test_list[1].append(0), which appends zero to all of the lists.

An example to understand test_list[1] = test_list[1] + [0]:

lst = [0]
print(lst) 
# output: [0]

lst += [1]
print(lst) 
# output: [0, 1]

lst.append(2)
print(lst) 
# output: [0, 1, 2]

lst2 = lst + [3]
print(lst) 
# output: [0, 1, 2]
# the lst did not change!

print(lst2) 
# output: [0, 1, 2, 3]
# a new instance of a list which is not lst

Now, let's see the object ids of the elements:

test_list = [[""]]*5
print(test_list)
# output: [[''], [''], [''], [''], ['']]

for i in range(5):
    object_id = id(test_list[i])
    print(object_id)
# output: 4405988168
# output: 4405988168
# output: 4405988168
# output: 4405988168
# output: 4405988168

test_list[1] = test_list[1] + [0]
print(test_list)
# output: [[''], ['', 0], [''], [''], ['']]

for i in range(5):
    object_id = id(test_list[i])
    print(object_id)
# output: 4405988168
# output: 4417503368     !!!!
# output: 4405988168
# output: 4405988168
# output: 4405988168

Explanation of python reference behavior when creating list of lists

2 Answers2