Where does PyCharm store compiled files?

Question

I'm new to PyCharm/Python, and can't figure out where the IDE stores compiled python *.pyc files.

Coming from the IntelliJ world, it is strange that I don't see any menu options to re-build the project, or build individual files.

I'm also unable to find any pyc files while searching the project directory, so basically, I've no idea whether successful compilation has happened at all, although the GitHub imported project is error free.

What can I do here?

chthonicdaemon · Accepted Answer · 2018-04-13T16:03:18.277

8

Because most Python implementations are interpreted rather than a compiled, the compilation step happens when you run the code. This is why the PyCharm UI features a prominent "Run" button (▶️) but no compile button.

It is true that for CPython there is a compilation step which compiles from the Python code to bytecode, but this is an implementation detail. CPython 3 stores its cached compilation results in .pyc files in a directory called __pycache__. These files are automatically generated when a module is imported (using import module will result in a module.pyc file) but not when a normal program is run.

Lastly, as per @shmee's comment, it is possible to compile a source file with the py_compile module, but I should emphasise that this is not usually done or necessary.

Now, if you are worried about checking that your code is correct, in the interpreted language world we rely more strongly on testing. I would recommend that you investigate tests for your code (using pytest and the excellent test integration in PyCharm).

edited Apr 13 '18 at 16:03

answered Apr 13 '18 at 15:39

chthonicdaemon

19,180
2
52
66

Actually, you can simply compile source files if you wish to: [py_compile](https://docs.python.org/3/library/py_compile.html). Also worth mentioning that the interpreter only compiles modules upon import. So if you have a module that is never imported by another one, it will never have a .pyc by default – shmee Apr 13 '18 at 15:43
@shmee I had already mentioned when compilation happens for CPython. I've also added a note about py_compile, thanks. – chthonicdaemon Apr 13 '18 at 15:47
Nitpick: *languages* are not compiled or interpreted, language *implementations* are. – juanpa.arrivillaga Apr 13 '18 at 15:59
@juanpa.arrivillaga I've reworded, but I must say it is very common to call talk about [interpreted languages](https://en.wikipedia.org/wiki/Interpreted_language). Do you know of a compiled Python implementation? I feel like the dynamic nature of the language wouldn't lend itself to compilation. – chthonicdaemon Apr 13 '18 at 16:05
It may be stretching the definition of "implementation" but [Cython](http://cython.org/) could be classified as "Python compiler". – MSeifert Apr 13 '18 at 16:08
Well, this is outside my area of expertise, but it should be compilable in principle, however, the dynamic nature would make it hard for the compiler to perform much of any optimizations – juanpa.arrivillaga Apr 13 '18 at 16:09
Classification is illusive. I used to use the signature "languages aren't fast, implementations are efficient", so I agree it's a nuanced point. Hope the current wording is specific enough. – chthonicdaemon Apr 13 '18 at 16:13
What do you mean by "These files are automatically generated when a module is imported but not when a normal program is run."? – variable Mar 05 '20 at 13:58

score 2 · Answer 2 · edited Jun 20 '20 at 09:12

Let me begin with a bit on terminology:

Python is a programming language. It's "just" the programming language specification.
CPython is the reference implementation of the Python language. It's actually just one of several different Python interpreters. CPython itself works (let's call it an implementation detail) by translating (but you could also say compiling) the code in imported Python files/modules to bytecode and then executing that bytecode. It actually stores the translation as .pyc files in the folder of that file) to make subsequent imports faster, but that's specific to CPython and can also be disabled.
PyCharm is an integrated development environment. However it requires to "Configure a Python Interpreter" to run Python code.

That means that PyCharm isn't responsible for creating .pyc files. If you configured a non-CPython interpreter or used the environmental variable to disable the pyc file creation there won't be any pyc files.

But if you used an appropriate CPython interpreter in PyCharm it will create .pyc files for the files/modules you successfully imported. That means you actually have to import or otherwise run the Python files in your project to get the .pyc files.

Actually the Python documentation contains a note about the "compiled" Python files:

To speed up loading modules, Python caches the compiled version of each module in the __pycache__ directory under the name module.version.pyc, where the version encodes the format of the compiled file; it generally contains the Python version number. For example, in CPython release 3.3 the compiled version of spam.py would be cached as __pycache__/spam.cpython-33.pyc. This naming convention allows compiled modules from different releases and different versions of Python to coexist.

Python checks the modification date of the source against the compiled version to see if it’s out of date and needs to be recompiled. This is a completely automatic process. Also, the compiled modules are platform-independent, so the same library can be shared among systems with different architectures.

Python does not check the cache in two circumstances. First, it always recompiles and does not store the result for the module that’s loaded directly from the command line. Second, it does not check the cache if there is no source module. To support a non-source (compiled only) distribution, the compiled module must be in the source directory, and there must not be a source module.

Some tips for experts:

You can use the -O or -OO switches on the Python command to reduce the size of a compiled module. The -O switch removes assert statements, the -OO switch removes both assert statements and doc strings. Since some programs may rely on having these available, you should only use this option if you know what you’re doing. “Optimized” modules have an opt- tag and are usually smaller. Future releases may change the effects of optimization.

A program doesn’t run any faster when it is read from a .pyc file than when it is read from a .py file; the only thing that’s faster about .pyc files is the speed with which they are loaded.

The module compileall can create .pyc files for all modules in a directory.

There is more detail on this process, including a flow chart of the decisions, in PEP 3147.

Per my understanding, the compilation to bytecode is a necessary part of the CPython runtime, however, the saving of `.pyc` files is optional to make startup time faster — juanpa.arrivillaga, Apr 13 '18 at 16:02
@juanpa.arrivillaga Yeah, I think so too. I meant the saving of the pyc file that can be disabled. — MSeifert, Apr 13 '18 at 16:03

Where does PyCharm store compiled files?

2 Answers2