78

If we take b = [1,2,3] and if we try doing: b+=(4,)

It returns b = [1,2,3,4], but if we try doing b = b + (4,) it doesn't work.

b = [1,2,3]
b+=(4,) # Prints out b = [1,2,3,4]
b = b + (4,) # Gives an error saying you can't add tuples and lists

I expected b+=(4,) to fail as you can't add a list and a tuple, but it worked. So I tried b = b + (4,) expecting to get the same result, but it didn't work.

M.K
  • 1,464
  • 2
  • 24
  • 46
  • 4
    I believe an answer can be found [here](https://stackoverflow.com/questions/13904493/is-the-behaviour-of-pythons-list-iterable-documented-anywhere). – jochen Oct 06 '19 at 17:37
  • 1
    or https://stackoverflow.com/questions/9897070/why-cant-i-add-a-tuple-to-a-list-with-the-operator-in-python – splash58 Oct 06 '19 at 17:38
  • At first I misread this and tried to close it as too broad, then retracted it. Then I thought it had to be a duplicate, but not only could I not re-cast a vote, I pulled my hair out trying to find other answers like those. :/ – Karl Knechtel Oct 06 '19 at 17:45
  • Very similar question: https://stackoverflow.com/questions/58048664/understanding-pythons-builtins-operator-overloading-behavior – sanyassh Oct 06 '19 at 18:44
  • Does this answer your question? [Different behaviour for list.\_\_iadd\_\_ and list.\_\_add\_\_](https://stackoverflow.com/questions/9766387/different-behaviour-for-list-iadd-and-list-add) – Pythoneer Jan 24 '23 at 17:11
  • Same question: https://stackoverflow.com/questions/9766387/different-behaviour-for-list-iadd-and-list-add – Pythoneer Jan 24 '23 at 17:11

7 Answers7

72

The problem with "why" questions is that usually they can mean multiple different things. I will try to answer each one I think you might have in mind.

"Why is it possible for it to work differently?" which is answered by e.g. this. Basically, += tries to use different methods of the object: __iadd__ (which is only checked on the left-hand side), vs __add__ and __radd__ ("reverse add", checked on the right-hand side if the left-hand side doesn't have __add__) for +.

"What exactly does each version do?" In short, the list.__iadd__ method does the same thing as list.extend (but because of the language design, there is still an assignment back).

This also means for example that

>>> a = [1,2,3]
>>> b = a
>>> a += [4] # uses the .extend logic, so it is still the same object
>>> b # therefore a and b are still the same list, and b has the `4` added
[1, 2, 3, 4]
>>> b = b + [5] # makes a new list and assigns back to b
>>> a # so now a is a separate list and does not have the `5`
[1, 2, 3, 4]

+, of course, creates a new object, but explicitly requires another list instead of trying to pull elements out of a different sequence.

"Why is it useful for += to do this? It's more efficient; the extend method doesn't have to create a new object. Of course, this has some surprising effects sometimes (like above), and generally Python is not really about efficiency, but these decisions were made a long time ago.

"What is the reason not to allow adding lists and tuples with +?" See here (thanks, @splash58); one idea is that (tuple + list) should produce the same type as (list + tuple), and it's not clear which type the result should be. += doesn't have this problem, because a += b obviously should not change the type of a.

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
  • 2
    Oof, quite right. And lists don't use `|`, so that kinda ruins my example. If I think of a clearer example later I'll swap it in. – Karl Knechtel Oct 07 '19 at 07:32
  • 1
    Btw `|` for sets is a commuting operator but `+` for lists is not. For that reason I don't think the argument about type ambiguity is particularly strong. Since the operator does not commute why require the same for types? One could just agree that the result has the l.h.s.'s type. On the other hand, by restricting `list + iterator`, the developer is encouraged to be more explicit about their intent. If you want to create a new list that contains the stuff from `a` extended by the stuff from `b` there's already a way to do this: `new = a.copy(); new += b`. It's one more line but crystal clear. – a_guest Oct 07 '19 at 11:11
  • The reason why `a += b` behaves differently than `a = a + b` is not efficiency. It's actually that Guido deemed the chosen behaviour _less_ confusing. Imagine a function receiving a list `a` as argument and then doing `a += [1, 2, 3]`. This syntax certainly _looks_ like it's modifying the list in place, rather than creating a new list, so the decision was made that it should behave according to what most people's intuition about the expected result. However, the mechanism also had to work for immutable types like `int`s, which led to the current design. – Sven Marnach Oct 10 '19 at 19:17
  • I personally think that the design is actually _more_ confusing than simply making `a += b` a shorthand for `a = a + b`, as Ruby did, but I can understand how we got there. – Sven Marnach Oct 10 '19 at 19:19
22

They are not equivalent:

b += (4,)

is shorthand for:

b.extend((4,))

while + concatenates lists, so by:

b = b + (4,)

you're trying to concatenate a tuple to a list

Nir Alfasi
  • 53,191
  • 11
  • 86
  • 129
15

When you do this:

b += (4,)

is converted to this:

b.__iadd__((4,)) 

Under the hood it calls b.extend((4,)), extend accepts an iterator and this why this also work:

b = [1,2,3]
b += range(2)  # prints [1, 2, 3, 0, 1]

but when you do this:

b = b + (4,)

is converted to this:

b = b.__add__((4,)) 

accept only list object.

Charif DZ
  • 14,415
  • 3
  • 21
  • 40
4

From the official docs, for mutable sequence types both:

s += t
s.extend(t)

are defined as:

extends s with the contents of t

Which is different than being defined as:

s = s + t    # not equivalent in Python!

This also means any sequence type will work for t, including a tuple like in your example.

But it also works for ranges and generators! For instance, you can also do:

s += range(3)
Acorn
  • 24,970
  • 5
  • 40
  • 69
3

The "augmented" assignment operators like += were introduced in Python 2.0, which was released in October 2000. The design and rationale are described in PEP 203. One of the declared goals of these operators was the support of in-place operations. Writing

a = [1, 2, 3]
a += [4, 5, 6]

is supposed to update the list a in place. This matters if there are other references to the list a, e.g. when a was received as a function argument.

However, the operation can't always happen in place, since many Python types, including integers and strings, are immutable, so e.g. i += 1 for an integer i can't possibly operate in place.

In summary, augmented assignment operators were supposed to work in place when possible, and create a new object otherwise. To facilitate these design goals, the expression x += y was specified to behave as follows:

  • If x.__iadd__ is defined, x.__iadd__(y) is evaluated.
  • Otherwise, if x.__add__ is implemented x.__add__(y) is evaluated.
  • Otherwise, if y.__radd__ is implemented y.__radd__(x) is evaluated.
  • Otherwise raise an error.

The first result obtained by this process will be assigned back to x (unless that result is the NotImplemented singleton, in which case the lookup continues with the next step).

This process allows types that support in-place modification to implement __iadd__(). Types that don't support in-place modification don't need to add any new magic methods, since Python will automatically fall back to essentially x = x + y.

So let's finally come to your actual question – why you can add a tuple to a list with an augmented assignment operator. From memory, the history of this was roughly like this: The list.__iadd__() method was implemented to simply call the already existing list.extend() method in Python 2.0. When iterators were introduced in Python 2.1, the list.extend() method was updated to accept arbitrary iterators. The end result of these changes was that my_list += my_tuple worked starting from Python 2.1. The list.__add__() method, however, was never supposed to support arbitrary iterators as the right-hand argument – this was considered inappropriate for a strongly typed language.

I personally think the implementation of augmented operators ended up being a bit too complex in Python. It has many surprising side effects, e.g. this code:

t = ([42], [43])
t[0] += [44]

The second line raises TypeError: 'tuple' object does not support item assignment, but the operation is successfully performed anywayt will be ([42, 44], [43]) after executing the line that raises the error.

Sven Marnach
  • 574,206
  • 118
  • 941
  • 841
  • Bravo! Having a reference to the PEP is especially useful. I added a link on the other end, to a previous SO question about the list-in-tuple behaviour. When I look back at what Python was like before 2.3 or so, it seems practically unusable compared to today... (and I still have a vague memory of trying and failing to get 1.5 to do anything useful on a very old Mac) – Karl Knechtel Oct 11 '19 at 21:48
2

Most people would expect X += Y to be equivalent to X = X + Y. Indeed, the Python Pocket Reference (4th ed) by Mark Lutz says on page 57 "The following two formats are roughly equivalent: X = X + Y , X += Y". However, the people who specified Python did not make them equivalent. Possibly that was a mistake which will result in hours of debugging time by frustrated programmers for as long as Python remains in use, but it's now just the way Python is. If X is a mutable sequence type, X += Y is equivalent to X.extend( Y ) and not to X = X + Y.

zizzler
  • 365
  • 1
  • 3
  • 7
  • > Possibly that was a mistake which will result in hours of debugging time by frustrated programmers for as long as Python remains in use < - did you actually suffer because of this? You seem to be talking from experience. I'd very much like to hear your story. – Veky Oct 07 '19 at 16:38
1

As it's explained here, if array doesn't implement __iadd__ method, the b+=(4,) would be just a shorthanded of b = b + (4,) but obviously it's not, so array does implement __iadd__ method. Apparently the implementation of __iadd__ method is something like this:

def __iadd__(self, x):
    self.extend(x)

However we know that the above code is not the actual implementation of __iadd__ method but we can assume and accept that there's something like extend method, which accepts tupple inputs.

Hamidreza
  • 1,465
  • 12
  • 31