18

Once in a while I run into a very difficult-to-debug problem: there's a leftover .pyc file somewhere in my $PYTHONPATH, and the matching .py file has been moved to somewhere else that's later in $PYTHONPATH - so when I try to import the module, the "orphaned" .pyc file is used and all changes to the "real" .py file are ignored, leaving me incredibly confused until I figure out that's what's happening.

Is there any way of making python not use "orphaned" .pyc files, or print a warning when using them?
Alternatively, does the fact that I have this problem at all mean I'm doing something wrong, and if so, what?

weronika
  • 2,561
  • 2
  • 24
  • 30
  • 2
    That will NOT remove pyc files from sub directories (shell expands the *.pyc first before the rm command sees the arguments). use `find * -name '*.pyc' | xargs rm -f` instead. Note 1. I use `find *` rather than `find .` so that .git etc isn't searched, 2. I delete all pyc files since python appears to only check the source is older than the compiled files, rather than matches the mod time exactly (since it doesn't make the mtime match when creating pyc files - under linux at least). – iheggie Nov 21 '14 at 03:01
  • 1
    `find . -name '*pyc' -delete` - apparently `find` has a flag for deleting found files – vlad-ardelean May 06 '15 at 08:02

2 Answers2

6

Try this (see here):

PYTHONDONTWRITEBYTECODE

If this is set, Python won’t try to write .pyc or .pyo files on the import of source modules. This is equivalent to specifying the -B option.

New in version 2.6.

But there is one issue associated with it: you will loose the benefit of having .pyc files (thus losing performance). The better, cleaner and friendlier way is to walk through the directory and clean orphaned .pyc files you do not need. You should do it with a script and be sure you do not have .pyc files that are not meant to be associated with .py ones (eg. for some level of obfuscation).

zizozu
  • 501
  • 4
  • 9
  • This prevents the *writing* of new .pyc files, but does it prevent python *loading* them? – Martijn Pieters May 28 '12 at 19:13
  • 1
    This does not prevent Python from loading *existing* .pyc files. I just tested this with Python 2.6.8 and 2.7.3 – Martijn Pieters May 28 '12 at 19:17
  • 1
    I was afraid those were my only options... Good to know about the PYTHONDONTWRITEBYTECODE env variable, though! I may use that for now (after removing all existing .pyc files), until I get around to writing a cleaning script and sticking it in my cron or something. – weronika May 28 '12 at 20:06
6

You cannot prevent Python from loading .pyc files if it finds them in it's path, no.

You have a few options:

  1. Import the offending module and figure out what it's path is:

    >>> import borkenmod
>>> import sys
>>> sys.modules[borkenmod.__name__].__file__
'lib/python2.7/borkenmod.pyc'

  2. Use a script to walk your PYTHONPATH and delete stale bytecode files. The linked script removes all .pyc files for which there is no corresponding .py file present.

  3. Wipe all .pyc files in your PYTHONPATH, and have Python regenerate them with the compileall module:

    python -m compileall [path]

    then remove that one file.

Do note that the last two options could result in the removal of legitimate python modules! Some python modules (especially commercially licenced) are distributed as compiled bytecode only.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Wipe+compileall is probably overkill - I should probably just write something to only wipe the orphaned ones. Thanks for the sys.modules trick! That should come in useful - at least once I figure out the wrong file is being loaded. – weronika May 28 '12 at 20:10