11

In python, we can concatenate lists in two ways:

  1. lst.extend(another_lst)
  2. lst += another_lst

I thought extend would be faster than using +=, because it reuses the list instead of creating a new one using the other two.

But when I test it out with timeit, it turns out that += is faster,

>>> timeit('l.extend(x)', 'l = range(10); x = range(10)')
0.16929602623
>>> timeit('l += x', 'l = range(10); x = range(10)')
0.15030503273
>>> timeit('l.extend(x)', 'l = range(500); x = range(100)')
0.805264949799
>>> timeit('l += x', 'l = range(500); x = range(100)')
0.750471830368

Is there something wrong with the code I put in timeit?

satoru
  • 31,822
  • 31
  • 91
  • 141
  • Can you show us the times you get? Also, these operations are both in-place, so that would not explain any difference. – Björn Pollex Nov 14 '10 at 09:57
  • I think you should use larger lists, at that tiny size, even the slightest background activity could trash the result. – vichle Nov 14 '10 at 09:58
  • @Space_C0wb0y @vischle I get results consistent with what Satoru reports even with lists of 200+ elements. += is slightly, but consistently, faster. It's quite odd. – Nicholas Knight Nov 14 '10 at 09:59
  • @Nicholas: I tested it too and see the same thing. I would have thought they would have implemented one operation using the other. – Björn Pollex Nov 14 '10 at 10:00
  • 4
    Why `+=` needs to create a new list? Are you confusing it with `+`? – kennytm Nov 14 '10 at 10:03
  • @KennyTM Isn't it just a short way of saying `lst = lst + another_lst`. – satoru Nov 14 '10 at 10:06
  • @Satoru: No, `+=` can be overloaded independently from `+`. – kennytm Nov 14 '10 at 10:07
  • 1
    Can you use `L` or `'li` as names of example lists? `l` looks like `1` with some fonts. – tshepang Nov 14 '10 at 10:09
  • 2
    @Tshepang: you know you can configure font preferences in your browser, right? – SilentGhost Nov 14 '10 at 10:12
  • 3
    @Silent, am aware. But it's good practice to remove that need, especially because my settings are default ("allow pages to choose their own fonts"). – tshepang Nov 14 '10 at 10:23
  • 4
    Here's an answer explaining how the `+=` overloading works: http://stackoverflow.com/questions/2347265/what-does-plus-equals-do-in-python/2347423#2347423 – Scott Griffiths Nov 14 '10 at 10:36
  • @Tshepang: since you have the default settings, shouldn't your comment become a suggestion in Meta instead? – tzot Nov 14 '10 at 15:32
  • @TZ, we can't change the entire look of SO for the sake of people unwilling to type `li` or `L` instead of `l` right? Even more important, this isn't limited to just SO. Someone else will print this, and the two characters just look the same. Should we ask those to also change font? BTW, I got this advice from a Python book, and felt it was a useful guideline. – tshepang Nov 14 '10 at 20:34
  • @Tshepang: If your monospace font is displaying 'l' and '1' alike, your monospace font is _broken_, get a new one. Mine works just fine. – Nicholas Knight Nov 15 '10 at 01:17
  • @Nic, I notice a misunderstanding here. I didn't mean `l` and `1` look *exactly the same*. I meant they look similar... too similar. – tshepang Nov 15 '10 at 06:16
  • 1
    Possible duplicate of [Concatenating two lists - difference between '+=' and extend()](https://stackoverflow.com/questions/3653298/concatenating-two-lists-difference-between-and-extend) – jamesdlin Nov 30 '17 at 02:50

1 Answers1

18

EDIT: I've tested the performance and I can't replicate the differences to any significant level.


Here's the bytecode -- thanks to @John Machin for pointing out inconsistencies.

>>> import dis
>>> l = [1,2,3]
>>> m = [4,5,6]
>>> def f1(l, m):
...     l.extend(m)
...
>>> def f2(l,m):
...     l += m
...
>>> dis.dis(f1)
  2           0 LOAD_FAST                0 (l)
              3 LOAD_ATTR                0 (extend)
              6 LOAD_FAST                1 (m)
              9 CALL_FUNCTION            1
             12 POP_TOP
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE
>>> dis.dis(f2)
  2           0 LOAD_FAST                0 (l)
              3 LOAD_FAST                1 (m)
              6 INPLACE_ADD
              7 STORE_FAST               0 (l)
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE

Notice that extend uses a CALL_FUNCTION instead of an INPLACE_ADD. Any trivial performance differences can probably be put down to this.

Katriel
  • 120,462
  • 19
  • 136
  • 170
  • Not only attribute lookup, but also a function call. – Constantin Nov 14 '10 at 10:22
  • @Constantin, mostly attribute lookup I think. `INPLACE_ADD` just routes to whatever `__iadd__` method has been defined on the object. – aaronasterling Nov 14 '10 at 10:25
  • @katriealex, @Constantin, @aaronsterling: Sheesh. It's a LOAD_ATTR and a CALL_FUNCTION as opposed to an INPLACE_ADD **and a STORE_FAST** – John Machin Nov 14 '10 at 11:00
  • @katriealex: Adding to the confusion, your list was a global in one case and a local in the other. Consider giving minimal examples with identical overheads e.g. in this case `def f1(a, b): a.extend(b)` and `def f2(a, b): a += b` – John Machin Nov 14 '10 at 11:05
  • @John: True. Edited, thanks =0. I also can't replicate the performance differences to any significant degree. – Katriel Nov 14 '10 at 11:14
  • @katrielalex I'm getting it on my system to about the same ratio as OP did. – aaronasterling Nov 14 '10 at 11:50
  • @John, Why are you placing so much emphasis on the STORE_FAST in explaining the speed difference? It seems that if anything it would count against += as the role that it plays isn't even necessary with `extend`. – aaronasterling Nov 14 '10 at 11:51
  • @aaronsterling: "sheesh" and emphasis because none of you actually mentioned that it was a part of the difference in opcodes used. Of course factoring that in means opcodes used don't provide any clear explanation for any speed difference -- the existence of which is disputed and results of robust benchmarking of which haven't been shown. – John Machin Nov 14 '10 at 20:47