170

As we all know, the pythonic way to swap the values of two items a and b is

a, b = b, a

and it should be equivalent to

b, a = a, b

However, today when I was working on some code, I accidentally found that the following two swaps give different results:

nums = [1, 2, 4, 3]
i = 2
nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
print(nums)
# [1, 2, 4, 3]

nums = [1, 2, 4, 3]
i = 2
nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]
print(nums)
# [1, 2, 3, 4]

This is mind-boggling to me. Can someone explain to me what happened here? I thought in a Python swap the two assignments happen simultaneously and independently.

Shaun Han
  • 2,676
  • 2
  • 9
  • 29
  • 165
    After studying your code snippet, the best answer I can give is "don't do that." The order of operations is what makes the difference I think, but wow that's confusing. – nicomp Jun 27 '21 at 15:53
  • 16
    @nicomp that's not a terribly satisfying answer. I often find that knowing the reason why something works the way it does helps me in other related areas. – Mark Ransom Jun 27 '21 at 15:55
  • 90
    That's why I added it as a comment. – nicomp Jun 27 '21 at 15:55
  • 1
    But for integers I don;t see anything like this. ```a,b = b,a``` and ```b,a = a,b``` both give the same result. – Ram Jun 27 '21 at 16:00
  • 5
    @Ram the example that fails is using the swapped number as an index into the list. Check the answers. – Mark Ransom Jun 27 '21 at 16:02
  • 46
    "As we all know, the pythonic way to swap two items a and b is", no that's how you swap **two variables**. If you use complex expressions order of evaluation comes into play. – GACy20 Jun 28 '21 at 06:55
  • 2
    This is because there's a lot more happening than just `a,b = b,a` – QBrute Jun 28 '21 at 14:40
  • 2
    To achieve what I believe you were expecting, you could do instead `j = nums[i]-1; nums[i], nums[j] = nums[j], nums[i]` which is safer imho; but you figured that out I guess – user5417363 Jun 28 '21 at 17:39
  • 3
    @MarkRansom: This may be controversial, but I actually don't recommend mastering order of operations to the extent that code like this is readable. If it's not vital to your job, remaining willfully ignorant of order of operation considerations like this (beyond recognizing when they appear) also has value; it's harder to recognize bad code when you understand it immediately. – Brian Jun 29 '21 at 14:16
  • 2
    It is really your intention to use `nums[i]-1` as an index, rather than `i-1` for instance? – jcaron Jun 29 '21 at 16:02
  • @jcaron I was trying to swap index 2 and 3 so that ``[1, 2, 4, 3]`` could become ``[1, 2, 3, 4]``. However, since ``nums[2] == 4``, I thought it would be intersting to do it like ``nums[nums[2]-1]`` instead of simply doing ``nums[3]``, and then I found this strange behavior. – Shaun Han Jun 29 '21 at 17:05
  • 2
    Nothing strange at all, you are changing the index you are using in the middle of your “swap”… – jcaron Jun 29 '21 at 18:49
  • 2
    I wonder if this question should be renamed "Understand Python swapping: why is a, b = , not always equivalent to b, a = , ?". I don't know what the accepted thinking on renames is in the Stack Overflow community. As it is the title is click-baity and misleading. – David Winiecki Jun 29 '21 at 20:18
  • @jcaron It is strange to me because I thought a Python swap happens simultaneously and independently. – Shaun Han Jun 29 '21 at 20:28
  • 1
    As you found it, it's not actually a swap operation. It's just an assignment with an intermediate copy in the middle. – jcaron Jun 29 '21 at 21:18
  • 1
    @DavidWiniecki: That wouldn't have the problem, you need to ", = a, b" in the title because it happens when the left side is complex, right side doesn't matter. – Ben Voigt Jun 29 '21 at 21:48
  • I agree that this question needs a rename. Nothing about this question is related to swapping like `a, b = b, a`. It's name should be along the lines of "Order of operation on assignment", potentially referencing tuple unpacking. Same with the tags on this question. – Gloweye Jun 30 '21 at 10:45
  • 1
    I can say as experienced programmer - your code is too complicated - if there is no super performance requirements do not write to complex code - it hard to read and analyze later and lead to bugs since not readable. I can write much more complicated code but I try to be not fat :) – Chameleon Jun 30 '21 at 13:03
  • Just curious, what were you working on when you discovered this behavior? It looks like the sort of swap you'd do if you were playing around with bubble sort. – Wayne Conrad Jun 30 '21 at 14:01
  • Wouldn't a swap be like `nums[i], nums[i+1] = nums[i+1], nums[i]` ? That code does not look like a swap. – lilole Jul 07 '21 at 18:25
  • Interesting. [This other question](https://stackoverflow.com/q/61415140) and [this answer](https://stackoverflow.com/a/41027438) both use the exact same variable names as this question. – John Y Jul 08 '21 at 18:44
  • Does this answer your question? [Weird behavior swapping two array elements while indexing by elements](https://stackoverflow.com/questions/61415140/weird-behavior-swapping-two-array-elements-while-indexing-by-elements) – mlibby Jul 10 '21 at 12:36

8 Answers8

142

From python.org

Assignment of an object to a target list, optionally enclosed in parentheses or square brackets, is recursively defined as follows.

...

  • Else: The object must be an iterable with the same number of items as there are targets in the target list, and the items are assigned, from left to right, to the corresponding targets.

So I interpret that to mean that your assignment

nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]

is roughly equivalent to

tmp = nums[nums[i]-1], nums[i]
nums[i] = tmp[0]
nums[nums[i] - 1] = tmp[1]

(with better error-checking, of course)

whereas the other

nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]

is like

tmp = nums[i], nums[nums[i]-1]
nums[nums[i] - 1] = tmp[0]
nums[i] = tmp[1]

So the right-hand side is evaluated first in both cases. But then the two pieces of the left-hand side are evaluated in order, and the assignments are done immediately after evaluation. Crucially, this means that the second term on the left-hand side is only evaluated after the first assignment is already done. So if you update nums[i] first, then the nums[nums[i] - 1] refers to a different index than if you update nums[i] second.

Silvio Mayolo
  • 62,821
  • 6
  • 74
  • 116
  • 53
    As a simpler example: If you have `a = [2, 2, 2, 2, 2]` and `b = 2` then `a[b], b = 3, 4; print(a)` should print `[2, 2, 3, 2, 2]` because `b` becomes `4` after `a[b]` is updated to `3` but `b, a[b] = 4, 3; print(a)` should print `[2, 2, 2, 2, 3]` because `b` becomes `4` before `a[b]` is updated to `3`. – Stobor Jun 28 '21 at 06:56
  • @Shaun The important part is that the earlier assignment has to modify the index of the latter assignment operation. Given that returning lvalues from function doesn't (or does it?) work in Python, this presumably means the only option for this is this array trickery. – Voo Jun 29 '21 at 11:13
  • 2
    Either array trickery or insane amounts of `__getattr__` / `__setattr__` fun. Array trickery is probably ten times easier, though. – Silvio Mayolo Jun 29 '21 at 13:10
  • Your last sentence is the key to the whole answer. Perfect. – simpleuser Jul 02 '21 at 00:22
  • 1
    The swap operation order may reversed in a different python implementation causing the bug to revive, so best to do all potential side-effect swaps explicitly! – Infernoz Jul 08 '21 at 23:40
72

This is because evaluation -- specifically at the left side of the = -- happens from left to right:

nums[i], nums[nums[i]-1] =

First nums[i] gets assigned, and then that value is used to determine the index in the assignment to nums[nums[i]-1]

When doing the assignment like this:

nums[nums[i]-1], nums[i] =

... the index of nums[nums[i]-1] is dependent on the old value of nums[i], since the assignment to nums[i] still follows later...

trincot
  • 317,000
  • 35
  • 244
  • 286
  • 5
    The array has been mutated. Using values from the mutated array as an index into the array will have results that depend upon the order of execution of the mutations. – Bae Jun 29 '21 at 03:29
  • 3
    @user253751, yes, but the OP's problem was not so much with the RHS. The RHS has already been evaluated when the *assignments* (on the LHS) start to be done. My answer focusses on that left side assignment sequence. – trincot Jun 29 '21 at 16:54
34

This happens according to the rules:

  • The right hand side is evaluated first
  • Then, each value of the left hand side gets its new value, from left to right.

So, with nums = [1, 2, 4, 3], your code in the first case

nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]

is equivalent to:

nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]

nums[2], nums[nums[2]-1] = nums[3], nums[2]

nums[2], nums[nums[2]-1] = 3, 4

and as the right hand side is now evaluated, the assignments are equivalent to:

nums[2] = 3
nums[nums[2]-1] = 4

nums[2] = 3
nums[3-1] = 4

nums[2] = 3
nums[2] = 4

which gives:

print(nums)
# [1, 2, 4, 3]

In the second case, we get:

nums[nums[2]-1], nums[2] = nums[2], nums[nums[2]-1]

nums[nums[2]-1], nums[2] = nums[2], nums[3]

nums[nums[2]-1], nums[2] = 4, 3

nums[nums[2]-1] = 4
nums[2] = 3

nums[4-1] = 4
nums[2] = 3

nums[3] = 4
nums[2] = 3
print(nums)
# [1, 2, 3, 4]
Thierry Lathuille
  • 23,663
  • 10
  • 44
  • 50
11

On the left hand side of your expression you are both reading and writing nums[i], I dunno if python gaurantees processing of unpacking operations in left to right order, but lets assume it does, your first example would be equivilent to.

t = nums[nums[i]-1], nums[i]  # t = (3,4)
nums[i] = t[0] # nums = [1,2,3,3]
n = nums[i]-1 # n = 2
nums[n] = t[1] # nums = [1,2,4,3]

While your second example would be equivilent to

t = nums[i], nums[nums[i]-1]  # t = (4,3)
n = nums[i]-1 # n = 3
nums[n] = t[0] # nums = [1,2,4,4]
nums[i] = t[0] # nums = [1,2,3,4]

Which is consistent with what you got.

plugwash
  • 9,724
  • 2
  • 38
  • 51
7

To understand the order of evaluation I made a 'Variable' class that prints when sets and gets occur to it's 'value'.

class Variable:
    def __init__(self, name, value):
        self._name = name
        self._value = value

    @property
    def value(self):
        print(self._name, 'get', self._value)
        return self._value

    @value.setter
    def value(self):
        print(self._name, 'set', self._value)
        self._value = value

a = Variable('a', 1)
b = Variable('b', 2)

a.value, b.value = b.value, a.value

When run results in:

b get 2
a get 1
a set 2
b set 1

This shows that the right side is evaluated first (from left to right) and then the left side is evaluated (again, from left to right).

Regarding the OP's example: Right side will evaluate to same values in both cases. Left side first term is set and this impacts the evaluation of the second term. It was never simultaneous and independently evaluated, it's just most times you see this used, the terms don't depended on each other. Setting a value in a list and then taking a value from the that list to use as an index in the same list is usually not a thing and if that's hard to understand, you understand it. Like changing the length of a list in a for loop is bad, this has the same smell. (Stimulating question though, as you may have guessed from me running to a scratch pad)

Guy Gangemi
  • 1,533
  • 1
  • 13
  • 25
4

One way to analyze code snippets in CPython is to disassemble its bytecode for its simulated stack machine.

>>> import dis
>>> dis.dis("nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]")
  1           0 LOAD_NAME                0 (nums)
              2 LOAD_NAME                0 (nums)
              4 LOAD_NAME                1 (i)

              6 BINARY_SUBSCR
              8 LOAD_CONST               0 (1)
             10 BINARY_SUBTRACT
             12 BINARY_SUBSCR
             14 LOAD_NAME                0 (nums)
             16 LOAD_NAME                1 (i)
             18 BINARY_SUBSCR

             20 ROT_TWO

             22 LOAD_NAME                0 (nums)
             24 LOAD_NAME                1 (i)
             26 STORE_SUBSCR

             28 LOAD_NAME                0 (nums)
             30 LOAD_NAME                0 (nums)
             32 LOAD_NAME                1 (i)
             34 BINARY_SUBSCR
             36 LOAD_CONST               0 (1)
             38 BINARY_SUBTRACT
             40 STORE_SUBSCR

             42 LOAD_CONST               1 (None)
             44 RETURN_VALUE

I added the blank lines to make reading this easier. The two fetch expressions are calculated in bytes 0-13 and 14-19. BINARY_SUBSCR replaces the top two values on the stack, an object and subscript, with the value fetched from the object. The two fetched values are swapped so that the first calculated is the first bound. The two store operations are done in bytes 22-27 and 28-41. STORE_SUBSCR uses and removes the top three values on the stack, a value to be stored, an object, and a subscript. (The return None part is apparently always added at the end.) The important part for the question is that the calculations for the stores are done sequentially in separate and independent batches.

The closest description in Python of the CPython calculation requires introduction of a stack variable

stack = []
stack.append(nums[nums[i]-1])
stack.append(nums[i])
stack.reverse()
nums[i] = stack.pop()
nums[nums[i]-1] = stack.pop()

Here is the disassembly for the reversed statement

>>> dis.dis("nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]")
  1           0 LOAD_NAME                0 (nums)
              2 LOAD_NAME                1 (i)
              4 BINARY_SUBSCR

              6 LOAD_NAME                0 (nums)
              8 LOAD_NAME                0 (nums)
             10 LOAD_NAME                1 (i)
             12 BINARY_SUBSCR
             14 LOAD_CONST               0 (1)
             16 BINARY_SUBTRACT
             18 BINARY_SUBSCR

             20 ROT_TWO

             22 LOAD_NAME                0 (nums)
             24 LOAD_NAME                0 (nums)
             26 LOAD_NAME                1 (i)
             28 BINARY_SUBSCR
             30 LOAD_CONST               0 (1)
             32 BINARY_SUBTRACT
             34 STORE_SUBSCR

             36 LOAD_NAME                0 (nums)
             38 LOAD_NAME                1 (i)
             40 STORE_SUBSCR

             42 LOAD_CONST               1 (None)
             44 RETURN_VALUE
Terry Jan Reedy
  • 18,414
  • 3
  • 40
  • 52
1

It seems to me that this would only happen when the contents of the list is in the range of list indexes for the list. If for example:

nums = [10, 20, 40, 30]

The code will fail with:

>>> nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

So definitely, a gotcha. Never Ever Use the Contents of a list as an index into that list.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
0

Thierry did give a good answer, let me be more clear. Be aware that if nums = [1, 2, 4, 3],

in this code:

nums[nums[i]-1], nums[i]
  • i is 2,
  • nums[nums[i]-1] is nums[4-1], so nums[3], (value is 3)
  • nums[i] is nums[2], (value is 4)
  • the result is: (3, 4)

in this code:

nums[i], nums[nums[i]-1]
  • nums[i] is nums[2] becomes 3, (=>[1, 2, 3, 3])
  • but nums[nums[i]-1] is not nums[4-1] but nums[3-1], so nums[2] too, becomes (back to) 4 (=>[1, 2, 4, 3])

Perhaps the good question concerning a swap, was using:

nums[i], nums[i-1] = nums[i-1], nums[i] ?

Try it:

>>> print(nums)
>>> [1, 2, 4, 3]
>>> nums[i], nums[i-1] = nums[i-1], nums[i]
>>> print(nums)
>>> [1, 4, 2, 3]

ChD

tontonCD
  • 320
  • 2
  • 6