2

I am using uWSGI to deploy my WSGI application. Are the Python file compiled for every request, or are they precompiled once? I don't see any .pyc files.

davidism
  • 121,510
  • 29
  • 395
  • 339
lmao
  • 452
  • 1
  • 5
  • 13

2 Answers2

2

Python caches modules bytecode - directly in the same location for python2.x, under a __pycache__ subfolder for python3 - but scripts (the difference is in usage - if you import it it's a module, if you execute it it's a script) are always recompiled (which is why main scripts are usually very very short an simple).

IOW, your main wsgi script will be recompiled once for each new server process. Typically a wsgi app is served as a long running process which will handle much more than one single request, so even then the initial compilation overhead is really not an issue (short script + only compiled once per process)...

Also, once a Python process is started, imported modules are cached in memory so they are only really imported (loaded) once per process.

Just note that the user under which the process is running must have write permissions on your app's directory in order to create the .pyc files... and of course read permissions on the .pyc files too.

bruno desthuilliers
  • 75,974
  • 6
  • 88
  • 118
  • 1
    I'm not questioning validity of your answer but could you point to the official documentation where this difference in behaviour for scripts is described? I did not found anything about it in [PEP-3147](https://www.python.org/dev/peps/pep-3147/#proposal) or in [Import. Cached Bytecode Invalidation](https://docs.python.org/3/reference/import.html#cached-bytecode-invalidation) – Alex Yu Mar 05 '19 at 12:40
  • 2
    @AlexYu this is at least mentionned here: https://docs.python.org/2/tutorial/modules.html#compiled-python-files and, in a slightly different way, here: https://docs.python.org/3/tutorial/modules.html#compiled-python-files – bruno desthuilliers Mar 05 '19 at 13:08
  • 1
    Thank you! I found in the first link for python-2: **"When a script is run by giving its name on the command line, the bytecode for the script is never written to a .pyc or .pyo file""**. It looks strange that this sentence is removed from python-3 documentation – Alex Yu Mar 05 '19 at 13:22
  • 2
    @AlexYu it has been replaced by "it always recompiles and does not store the result for the module that’s loaded directly from the command line", possibly to account for the `python -m modulename` CLI option too. – bruno desthuilliers Mar 05 '19 at 13:23
  • Thanks a lot @bruno & Alex. Now, I know why my main script should be small and that I should import modules to avoid repeated compilation.. – lmao Mar 07 '19 at 17:23
1

If you're using CPython then by default, no it does not compile every time a request is received unless you manually configure it to do this.

By the first request, the uWSGI will load the python scripts bytecode and will reload it in 2 different scenarios I know of:

  1. There is a new import that has not been loaded before (only the new module will be parsed and converted to bytecode).
  2. You explicitly ran service uwsgi restart

Yet, there is still a way to keep reloading python scripts in every request by suppressing it by the interpreter, example: python -B my_amazing_view.py

For more details check here:

  1. What is pycache
  2. Should I generate .pyc
Ramy M. Mousa
  • 5,727
  • 3
  • 34
  • 45
  • 1
    This is actually a bit more complicated. The Python runtime only compiles (or, more exactly, only cache the compiled code for) modules, not scripts (what makes the difference is actually whether you import the file or execute it). – bruno desthuilliers Mar 05 '19 at 12:02
  • Thanks a lot sir, your links did help a lot. – lmao Mar 07 '19 at 17:24