Why is appending to a string so much slower in PyPy than in CPython?

Question

I have this relatively simple Python program

inpt = "0"*1000000
out = ""

for x in inpt:
    out += x

When I run it using the default Python 2.7.9, it finishes relatively quickly:

time python te.py
0.27s user 0.02s system 93% cpu 0.308 total

When I run it using PyPy 5.6.0, it goes from taking less than half a second to taking nearly a minute and a half:

time pypy te.py
85.73s user 15.40s system 95% cpu 1:45.53 total

What could be causing this dramatic slowdown? I usually use PyPy to speed up my Python code, but this result surprises me.

Maybe related to http://stackoverflow.com/questions/4435169/good-way-to-append-to-a-string - don't use `+` to append to strings for this reason — mata, Dec 16 '16 at 08:56
@mata The answer there certainly is exactly the explanation; just the *question* is somewhat different unfortunately. — deceze, Dec 16 '16 at 08:58
Cpython has an implementation detail where string concatenation is optimized for simple cases (see [here](http://stackoverflow.com/questions/40996801/why-is-new-file-line-string-so-much-faster-than-new-file-new-file-li)) by mutating the string. I suspect this does not occur in PyPy. But you should be using the `str.join()` method anyway. — Chris_Rands, Dec 16 '16 at 09:00
@mata I disagree with the duplicate. The other post solves the _how to_. However, in the OP it is being asked the _why_ — Jalo, Dec 16 '16 at 14:27
The accepted answer explains exactly why, cPython can optimize this to extend the original string instead of creating a new copy each time because by its use of refcounting it knows that the modified string is not in use anywhere else. pypy doesn't have such optimizations, your asking for a new copy in each iteration and that is what you get. — mata, Dec 16 '16 at 19:13
Try timing it while using the string in different places, CPython shouldn't optimize then. — xuiqzy, Apr 16 '21 at 11:40

0 Answers0