1

Is there an easy way to pre-"compile" the entire Python standard library to byte code? If so, would this also guarantee that Python never re-"compiles" a file if a corresponding .pyc file already exists (as long as the standard library is not updated or modified)?

Background: On an application server, Python sometimes spontaneously goes into a broken state that manifests itself like this:

C:\> python
Traceback (most recent call last):
  File "C:\Python27\lib\site.py", line 563, in <module>
    main()
  File "C:\Python27\lib\site.py", line 552, in main
    aliasmbcs()
  File "C:\Python27\lib\site.py", line 478, in aliasmbcs
    enc = locale.getdefaultlocale()[1]
AttributeError: 'module' object has no attribute 'getdefaultlocale'

and that can be resolved by re-installing Python. Someone else told me that he has seen corrupted (0-byte) .pyc files and that deleting those solved the problem. However, I cannot reproduce the issue by deliberately creating 0-byte .pyc files.

I don't like the fact that files belonging to the standard library are "compiled" to byte code on demand, as this happens within a process running both Python and native code that can crash unexpectedly. I would prefer to have all Python code pre-compiled, whether or not it helps to resolve the above situation.

Florian Winter
  • 4,750
  • 1
  • 44
  • 69
  • 1
    If that's the case you should file a bug with the Python folks. But in principle, the interpreter should recompile a module which has s damaged .pyc file the next time it is needed. – Horia Coman May 05 '17 at 07:42
  • 1
    I just tried to reproduce this by creating a 0-byte file `C:\Python27\Lib\hmac.pyc` and running `import hmac` in an interactive Python session. The file was recreated at a size of 5kb, and I did not get an error. I have the information about 0-byte Python files from a colleague. The actual error I see is that I get an exception whenever running Python, and that reinstalling Python resolves the problem. Whether it is caused by corrupted .pyc files is not known. I will update my question. – Florian Winter May 05 '17 at 08:00
  • According to http://stackoverflow.com/a/17030047/2279059, corruption of Python files may be possible in Python 2.x, whereas Python 3.3 has mechanisms to prevent corruption from happening. – Florian Winter May 05 '17 at 08:15
  • A snapshot of the Python folder contains a supposedly corrupted (1kb) `locale.pyc`. The file is supposed to be 49kb in size. Copying the "corrupted" file to a healthy Python installation does not reproduce the issue. Instead, the file is regenerated to 49kb in size. – Florian Winter May 05 '17 at 08:52

1 Answers1

1

Make the system .pyc files read-only, even for administrator. This should prevent Python from modyfing them unnecessarily.

If You're on Windows go to properties → security → edit, and deny all privileges for all groups.

user2622016
  • 6,060
  • 3
  • 32
  • 53
  • Confirming that this prevents `.pyc` files from being created, and that Python still works without the `.pyc` (tested by deleting `locale.pyc`, denying write permission, running `import locale` and `dir(locale)` and verifying that `locale.pyc` does not exist). However, I suspect that not having the byte code files can have an impact on performance, so I would like to make sure they are there before denying write access (which then also might be unnecessary). – Florian Winter May 05 '17 at 08:56