8

We've used Python 2 for an embedded application that we're currently prototyping on Raspberry Pi. It's been a bit slow, but acceptable.

Now we've ported the application to Python 3. But for some reason the application runs about 4 times slower than with Python 2! I was expecting Python 3 to be a bit slower, but nothing like this!

Our Linux distribution is based on Yocto 2.2 Morty and we are using the default Python 3.5.2 recipe with no customizations. We are also using the meta-raspberrypi layer for Raspberry compatibility.

I tried to time "pip3 --help" and got the following result:

$ time pip3 --help >/dev/null

real    0m22.686s
user    0m22.210s
sys     0m0.420s

Then I tried the same test on the same hardware (same sd card as well) using the latest Raspbian distro:

$ time pip3 --help >/dev/null

real    0m6.874s
user    0m6.530s
sys     0m0.310s

$ time pip --help >/dev/null

real    0m4.329s
user    0m4.000s
sys     0m0.300s

Python 2 pip is a little bit faster than Python 3, but the most important thing is that pip3 runs more than 3 times faster on Raspbian than on Yocto!

The tests are very repeatable, so they are not caused by pyc-generation or caching or something like that.

Any ideas how to get Yocto as fast as Raspbian?

Update:

There was some discussions about different versions of Python and Pip in Raspbian vs Yocto below, so I made a new benchmark using only Python standard libraries:

Yocto 2.2 Morty:

sh-4.3# time python3 --version
Python 3.5.2

real    0m0.024s
user    0m0.010s
sys     0m0.000s
sh-4.3# time python3 -c "import asyncio"

real    0m3.439s
user    0m3.300s
sys     0m0.110s

Raspbian:

pi@raspberrypi:~$ time python3 --version
Python 3.4.2

real    0m0.020s
user    0m0.000s
sys     0m0.010s
pi@raspberrypi:~$ time python3 -c "import asyncio"

real    0m2.023s
user    0m1.850s
sys     0m0.160s

I then downloaded Python 3.5.2 on Raspbian and built it from source, with no custom configuration (./configure; make; make install). With Python 3.5.2 I get the following result:

pi@raspberrypi:~$ time python3.5 --version
Python 3.5.2

real    0m0.018s
user    0m0.000s
sys     0m0.010s
pi@raspberrypi:~$ time python3.5 -c "import asyncio"

real    0m2.689s
user    0m2.610s
sys     0m0.070s

So it seems that Python 3.5 is considerably slower than 3.4, but still much, much slower on a default Yocto build than on Raspbian with the default build configuration.

Update 2:

I built Python 3.5.2 the same way on my Yocto system ("./configure; make; make install") and got about 20% improvement compared to the standard Python recipe:

root@la:/var/src/Python-3.5.2# time python3.5 -c "import asyncio"

real    0m2.914s
user    0m2.750s
sys     0m0.130s
Jonatan
  • 3,752
  • 4
  • 36
  • 47
  • Can you specify the versions of python(s), pip(s) and yocto? You mention yocto 2.0 and python 3.5.2 but those versions don't match... – Jussi Kukkonen Apr 05 '17 at 17:54
  • For reference, I tried pip and pip3 in a yocto build in qemu: They are about as fast but both seem quite slow to start up indeed -- about 10 secs just to print the version string. – Jussi Kukkonen Apr 05 '17 at 17:56
  • @jku you're right, i was picking that from my head and obviously had the wrong yocto version. It's Yocto 2.2 Morty. pip3 version is 8.1.2. Python is version 3.5.2. – Jonatan Apr 05 '17 at 18:00
  • Also, I should note that I have not tested with Python 2 on this Yocto version. Last time we built the distro with the application written in Python 2 was close to a year ago, so that's what I'm comparing with. Unfortunately I don't even have pip2 installed, but I can add it on next build for comparison if that is of any help. – Jonatan Apr 05 '17 at 18:08
  • Are the raspbian python3 and pip3 versions similar to the ones on your yocto build? If they are, then it might be worth filing a bug on bugzilla.yoctoproject.org – Jussi Kukkonen Apr 05 '17 at 19:36
  • @jku raspbians default is 3.4.2 and the pip versions are very different. But this is not related to just pip. I've updated the text with a new benchmark using only the standard python library and I've also built Python 3.5.2 on Raspbian for comparison. – Jonatan Apr 06 '17 at 11:05

1 Answers1

1

Yocto Python 2 is compiled in optimized mode. Try configuring Python 3 (3.5.3+ I think) with ./configure --enable-optimizations as also discussed here.

Community
  • 1
  • 1
brennan
  • 3,392
  • 24
  • 42
  • Python3 in Yocto 2.0 still had the optimization patch you refer to (that set Py_OptimizeFlag=1 by default). The patch was later removed as it doesn't really optimize anything -- the only meaningful thing it does is disabling asserts as far as I know. I'm interested in finding out what the issue is but I don't think it's related to this... – Jussi Kukkonen Apr 05 '17 at 17:10
  • A later comment verified this was on yocto 2.2 which I think does not have the optimization patch anymore -- so my comment above is not valid (my opinion stands but the python may really be running in "not optimized" mode). – Jussi Kukkonen Apr 05 '17 at 19:39