4

How can I change the priority of the path in sys.path in python 2.7? I know that I can use PYTHONPATH environment variable, but it is what I will get:

$ PYTHONPATH=/tmp python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for i in sys.path:
...   print i
... 

/usr/local/lib/python2.7/dist-packages/pycuda-2014.1-py2.7-linux-x86_64.egg
/usr/local/lib/python2.7/dist-packages/pytest-2.6.2-py2.7.egg
/usr/local/lib/python2.7/dist-packages/pytools-2014.3-py2.7.egg
/usr/local/lib/python2.7/dist-packages/py-1.4.24-py2.7.egg
/usr/lib/python2.7/dist-packages
/tmp
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages/PILcompat
/usr/lib/python2.7/dist-packages/gtk-2.0
/usr/lib/python2.7/dist-packages/ubuntu-sso-client
>>>

/tmp is added between /usr/lib/python2.7/dist-packages and /usr/lib/python2.7.

My goal is to make python to load packages from /usr/local/lib/python2.7/dist-packages first.

Here is what I want:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.version
<module 'numpy.version' from '/usr/local/lib/python2.7/dist-packages/numpy/version.pyc'>
>>>

If I install python-numpy by apt-get install python-numpy. Python will try to load from /usr/lib/python2.7 and not the one I compiled.

Bakuriu
  • 98,325
  • 22
  • 197
  • 231
wh0
  • 510
  • 1
  • 6
  • 19
  • From the outputs you provided python *is* loading modules from `/usr/local` before those in `/usr/lib` so, where's the problem? – Bakuriu Oct 04 '14 at 13:32
  • Have you tried simply editing the variable sys.path to see if that works? – John Zwinck Oct 04 '14 at 13:32
  • @Bakuriu no. I remove the installed numpy by "apt-get remove python-numpy" to get this result for asking the solution. – wh0 Oct 04 '14 at 13:33
  • @JohnZwinck Yes, but the change is not permanent. – wh0 Oct 04 '14 at 13:33
  • 1
    Well you could put it in your $PYTHONSTARTUP file, then it would be "permanent" in the same way that modifying $PYTHONPATH would be. – John Zwinck Oct 04 '14 at 13:34
  • 1
    Note that what you are seeing is due to the implicit [`import site`](https://docs.python.org/3.4/library/site.html) done during initialization. If you launch the interpreter with the `-S` options you'll see that `/tmp` is added at the beginning of the list instead of among the other paths. – Bakuriu Oct 04 '14 at 13:36
  • Thanks. $PYTHONSTARTUP seems work for me. – wh0 Oct 04 '14 at 13:37

2 Answers2

9

As you may know, sys.path is initialized from:

  • the current directory
  • your PYTHONPATH
  • an installation-dependent default

However unfortunately that is only part of the story: setuptools creates easy-install.pth files, which also modify sys.path and worst of all they prepend packages and therefore totally mess up the order of directories.

In particular (at least on my system), there is /usr/local/lib/python2.7/dist-packages/easy-install.pth with the following contents:

import sys; sys.__plen = len(sys.path)
/usr/lib/python2.7/dist-packages
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

This causes /usr/lib/python2.7/dist-packages to be prepended even before your PYTHONPATH!

What you could do is simply change the 2nd line in this file to

/usr/local/lib/python2.7/dist-packages

and you will get your desired priority.

However beware this file might be overwritten or changed again by a future setuptools invocation!

kynan
  • 13,235
  • 6
  • 79
  • 81
2

We encountered an almost identical situation and wanted to expand upon @kynan's response which is spot-on. In the case where you have such an easy-install.pth that you want to overcome, but which you cannot modify it (say you are a user with no root/admin access), you can do the following:

  • Set up an alternate python installation scheme

    • e.g. we use a PYTHON HOME install (setting PYTHONUSERBASE)

  • Create a user/home site-packages

    • You can do this by installing a package into the user env: pip install <package> --user

  • Create a pth to set sys.__egginsert to workaround the system/distribution easy-install.pth

    • Create a $PYTHONUSERBASE/lib/python2.7/site-packages/fix_easy_install.pth
    • Containing: import sys; sys.__egginsert = len(sys.path);

This will set sys.__egginsert to point to the end of your sys.path including your usersite paths. When the nefarious system/dist easy-install.pth will then insert its items at the end of the system path.

Richard Cantzler
  • 21
  • 1