What sets up sys.path with Python, and when?

Question

When I run

import sys 
print sys.path

on my Mac (Mac OS X 10.6.5, Python 2.6.1), I get the following results.

/Library/Python/2.6/site-packages/ply-3.3-py2.6.egg
...
/Library/Python/2.6/site-packages/ipython-0.10.1-py2.6.egg
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages

/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload
/Library/Python/2.6/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode

They are grouped into 5 categories.

• /Library/Python/2.6/site-packages/*.egg
• /Library/Python/2.6/site-packages
• Frameworks/Python.framework/Versions/2.6/lib/python2.6
• Frameworks/Python.framework/Versions/2.6/Extras/lib/python
• PATH from PYTHONPATH environment variable.

And I can add more paths using the code

sys.path.insert(0, MORE_PATH)

• What routines sets up those paths, and when?
• Are some of the paths are built in python source code?
• Is it possible that the paths inserted with 'sys.path.insert' are ignored? I'm curious about this, as with mod_wsgi, I found the paths are not found with 'sys.path.insert'. I asked another post for this question.

ADDED

Based on Michael's answer, I looked into site.py, and I got the following code.

def addsitepackages(known_paths):
    """Add site-packages (and possibly site-python) to sys.path"""
    sitedirs = []
    seen = []

    for prefix in PREFIXES:
        if not prefix or prefix in seen:
            continue
        seen.append(prefix)

        if sys.platform in ('os2emx', 'riscos'):
            sitedirs.append(os.path.join(prefix, "Lib", "site-packages"))
        elif sys.platform == 'darwin' and prefix == sys.prefix:
            sitedirs.append(os.path.join("/Library/Python", sys.version[:3], "site-packages"))

I also think that the directory name that has site.py (/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6 for my Mac) should be built into Python source code.

Answer 1

Most of the stuff is set up in Python's site.py which is automatically imported when starting the interpreter (unless you start it with the -S option). Few paths are set up in the interpreter itself during initialization (you can find out which by starting python with -S).

Additionally, some frameworks (like Django I think) modify sys.path upon startup to meet their requirements.

The site module has a pretty good documentation, a commented source code and prints out some information if you run it via python -m site.

Answer 2

From Learning Python:

sys.path is the module search path. Python configures it at program startup, automatically merging the home directory of the top-level file (or an empty string to designate the current working directory), any PYTHONPATH directories, the contents of any .pth file paths you've created, and the standard library directories. The result is a list of directory name strings that Python searches on each import of a new file.

Answer 3

site.py is indeed the answers. I wanted to remove any dependencies on the old Python that is installed by default on my mac. This works pretty good, as 'site.py' is called each time the python interpreter is started.

For Mac, I manually added the following line at the end of main() in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/site.py:

    sys.path =  filter (lambda a: not a.startswith('/System'), sys.path)

Answer 4

Path has these parts:

• OS paths that have your system libraries
• current directory python started from
• environmental variable $PYTHONPATH
• you can add paths at runtime.

Paths are not ignored. But, they may not be found and that will not raise an error. sys.path should only be added too, not subtracted from. Django would not remove paths.

Answer 5

Adding to the accepted answer, and addressing the comments that say a module shouldn't remove entries from sys.path:

This is broadly true but there are circumstances where you might want to modify sys.path by removing entries. For instance - and this is Mac-specific; *nix/Windows corollaries may exist - if you create a customised Python.framework for inclusion in your own project you may want to ignore the default sys.path entries that point at the system Python.framework.

You have a couple of options:

1. Hack the site.py, as @damirv indicates, or

2. Add your own sitecustomize module (or package) to the custom framework that achieves the same end result. As indicated in the site.py comments (for 2.7.6, anyway):

   After these path manipulations, an attempt is made to import a module named sitecustomize, which can perform arbitrary additional site-specific customizations. If this import fails with an ImportError exception, it is silently ignored.

Answer 6

Also note: if the PYTHONHOME env var is set, standard libraries will be loaded from this path instead of the default, as documented.

This is not a direct answer to the question, but something I just discovered that was causing the wrong standard libraries to be loaded, and my searches lead me here along the way.

Answer 7

You are using system python /usr/bin/python.

sys.path is set from system files at python startup.

Do not touch those files, in particular site.py, because this may perturb the system.

However, you can change sys.path within python, in particular, at startup :

in ~/.bashrc or ~/.zshrc:

export PYTHONSTARTUP=~/.pythonrc

in ~/.pythonrc:

write your changes to sys.path.

Those changes will be only for you in interactive shells.

For hacking at little risk for the system, install you own and more recent python version.