5

I have installed Pandas version 0.12.0 site-wide and a user needs 0.13.0. I told him to install it to his home directory which he did but when he types import pandas it finds the old module.

So I decided to print out his sys.path and noticed these paths in this order (others removed to keep this brief):

[
    '',
    '/apps/python/2.7.5/lib/python2.7/site-packages/pandas-0.12.0-py2.7-linux-x86_64.egg',
    '/home/user/.local/lib/python2.7/site-packages',
    '/apps/python/2.7.5/lib/python2.7/site-packages'

]

The PYTHONPATH is showing up after pandas as well:

[
    '',
    '/apps/python/2.7.5/lib/python2.7/site-packages/pandas-0.12.0-py2.7-linux-x86_64.egg',
    '/usr/lib64', // this is the PYTHONPATH
    '/home/user/.local/lib/python2.7/site-packages',
    '/apps/python/2.7.5/lib/python2.7/site-packages'

]

What would cause pandas-0.12.0 to be specifically loaded before anything else, even the PYTHONPATH? There are several other packages that exhibit the same behavior as well. All packages have been installed by pip or by doing python setup.py install; would either method cause such an issue? I don't think we've manually edited anything.

Levi Morrison
  • 19,116
  • 7
  • 65
  • 85
  • possible duplicate of [Eggs in path before PYTHONPATH environment variable](http://stackoverflow.com/questions/5984523/eggs-in-path-before-pythonpath-environment-variable) – Levi Morrison Jun 15 '14 at 15:00

1 Answers1

2

Generally, the module search path documentation lists the priorities of Python's import sources: https://docs.python.org/2/tutorial/modules.html#the-module-search-path. According to this, PYTHONPATH comes right after the current working directory.

Pretty sure the high priority of pip-installed packages comes from certain pth files in your site-packages directory (however, I was not able to quickly find out the priority of these pth files compared to PYTHONPATH). Have a look at https://docs.python.org/2/library/site.html which explains how these files work. There is an advice, which should solve your problem:

This module is automatically imported during initialization. The automatic import can be suppressed using the interpreter’s -S option.

PYTHONPATH survives -S (tested with Python 2.7.3):

$ export PYTHONPATH="FOO"
$ python -S
>>> import sys
>>> "FOO" in sys.path
True

I see that this solution is problematic in case you also need packages from paths set up by site.py. However, in that case careful adjustment of PYTHONPATH would still help.

Edit: this looks like a longstanding issue with easy_install/setuptools/distribute. It is known that these might prepend to sys.path, effectively overriding PYTHONPATH, which is often times undesired behavior, see https://bugs.launchpad.net/ubuntu/+source/distribute/+bug/821000. I am not sure whether current pip and distribute still have this bug, maybe you should update to the most recent version of these.

Dr. Jan-Philip Gehrcke
  • 33,287
  • 14
  • 85
  • 130