Appending varying values to loop results in list of last value only

Question

I am simulating a position moving randomly and want to create a list of the position. I create a start position randomly and then add steps of random size to this start position:

import numpy as np

pos_range=1E-6
position=pos_range*np.random.rand(2)

steps=10
step_size=20E-9;

position_list=[]

for i in range(0,steps):
    step=step_size*np.random.rand(2)
    position+=step
    print(position)
    position_list.append(position)

The result of the print is

[6.47220682e-07 2.84186976e-07]
[6.48019947e-07 2.89315337e-07]
[6.50324286e-07 2.96818080e-07]
[6.64316483e-07 3.10052685e-07]
[6.79662022e-07 3.20408865e-07]
[6.85052985e-07 3.31529075e-07]
[6.97773479e-07 3.45764518e-07]
[7.11152822e-07 3.46809336e-07]
[7.14484731e-07 3.54165996e-07]
[7.20412104e-07 3.58339358e-07]

which is my desired result. But position_list only contains the same data of the last position value in all rows:

    position_list
array([[7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07],
       [7.20412104e-07, 3.58339358e-07]])

I suspect it has something to with how the numpy array is stored in memory or the mixture of lists and numpy array. All the workarounds that I tried are tedious and I am interested why this code does not work as intended.

score 1 · Answer 1 · answered Mar 01 '22 at 09:02

1

Change

position_list.append(position)

To

position_list.append(list(position))

answered Mar 01 '22 at 09:02

Josip Juros

349
2
12

score 0 · Answer 2 · answered Mar 01 '22 at 09:08

0

position += step modifies position in place. You are appending the same object to position_list over and over again.

We can confirm this by looking at the set of unique object ids in position_list.

>>> set(map(id, position_list))
{1637236197296}

There is an easy fix. Change position += step to position = position + step. This avoids calling numpy.ndarray.__iadd__.

Also see Is i = i + n truly the same as i += n?

answered Mar 01 '22 at 09:08

timgeb

76,762
20
123
145

Very interesting explanation, Thank you, But why does `position_list.append(list(position))` work then? by this logic it should still append the same object but as a list, But with this edit it works properly? – Josip Juros Mar 01 '22 at 09:12
https://prnt.sc/grl7aq1Sm2xt Screenshot as proof – Josip Juros Mar 01 '22 at 09:13
@JosipJuros in that case you are appending a new list every time. Check `set(map(id, position_list))`. – timgeb Mar 01 '22 at 09:14
Yes but that new list is a cast of that same object? Ah I see. It just saves one ID, but when append a list every time every list has its own ID, hence why it saves properly? – Josip Juros Mar 01 '22 at 09:15
1

"Yes but that new list is a cast of that same object?" With the `position_list.append(list(position))` solution the object `position` is always the same, though its content changes during the loop. `list(position)` builds a new list from the `position` object in each iteration. The references inside the list always point to the same numbers as when it was constructed. – timgeb Mar 01 '22 at 09:18
@JosipJuros consider `l = [1, 2, 3]; ll = list(l); l[0] = 10; print(l, ll)`. – timgeb Mar 01 '22 at 09:20
1

I think I understand, Thank you very much for explaining it. – Josip Juros Mar 01 '22 at 09:20
1

@JosipJuros and with the `position = position + step` solution you are building a new position array in each iteration and then reassign the name `position`. – timgeb Mar 01 '22 at 09:23
2

It might make the code change more obvious if you write *position_list.append(position.copy())* – DarkKnight Mar 01 '22 at 09:26

Appending varying values to loop results in list of last value only

2 Answers2