5

When developing a Python web app (Flask/uWSGI) and running it on my local machine, *.pyc files are generated by the interpreter. My understanding is that these compiled files can make things load faster, but not necessarily run faster.

When I deploy this same app to production, it runs under a user account that has no write permissions on the local file system. There are no *.pyc files committed to source control, and no effort is made to generate them during the deploy. Even if Python wanted to write a .pyc file at runtime, it would not be able to.

Recently I started wondering if this has any tangible effect on the performance of the app, either in terms of the very first pageview after the process starts, or consistently throughout its entire lifetime.

Should I throw a python -m compileall in as part of my deploy scripts?

Guy Avraham
  • 3,482
  • 3
  • 38
  • 50
smitelli
  • 6,835
  • 3
  • 31
  • 53
  • As I see it, it's totally silly to deploy it, but you can do it, since Python has to be converted to `bytecode` before it can be interpreted, keeping `.pyc` files around can help speed up execution, since you don't have to recompile, here read this maybe it will help http://stackoverflow.com/questions/11368304/what-are-the-limitations-of-distributing-pyc-files – copser Feb 25 '16 at 05:07
  • 1
    On the other, if you have a long running application, like Flask, it will only reduce the startup time. Flask will rarely load modules while running. – Klaus D. Feb 25 '16 at 05:28

1 Answers1

5

Sure, you can go ahead and precompile to .pyc's as it won't hurt anything.

Will it affect the first or nth pageload? Assuming Flask/WSGI runs as a persistent process, not at all. By the time the first page has been requested, all of the Python modules will have already been loaded into memory (as bytecode). Thus, server startup time will be the only thing affected by not having the files pre-compiled.

However, if for some reason a new Python process is invoked for each page request, then yes, there would (probably) be a noticeable difference in performance and it would be better to pre-compile.

As Klaus said in the comments above, the only other time a pageload might be affected is if a function happens to try and import a module that hasn't already been imported. This will require the module to be parsed and converted to bytecode then loaded into memory before being able to continue.

OozeMeister
  • 4,638
  • 1
  • 23
  • 31
  • I'd just like to state a case where "it won't hurt" is not true: it doesn't work with projects using venusian, e.g. pyramid. Venusian completely ignores pyc/pyo files which renders at least every pyramid app infunctional (is that even a word?). – daten May 29 '20 at 21:05