Programmatically append to PYTHONPATH in VirtualEnv using .pth

Question

How can you use a Python script to programmatically append the current path to the current Virtualenv's PYTHONPATH, using either .pth or another method?

This needs to work on both Windows and Linux.

I'm trying to have a Python script setup.py in the project root path which when executed, adds the project root path to PYTHONPATH so other scripts in nested directories can import modules by relative import from the project root.

MyProjectRoot
+ setup.py
+ data 
+ source
    + foo
        + lib
            + qux
                + A.py
    + bar
        + assets
            + B.py
    + baz
        + C.py

For example, in C.py, we can use

import source.foo.lib.qux.A
import source.bar.assets.B

to avoid having to add a variant of the following to every file that wishes to use import relative to the project root:

basePath = os.path.abspath("../..")
sys.path.append(basePath)

If you just want to set up the environement for your project, I add stuff like this to the virtualenv "activate" script. You already have to run that to start the environment in the first place. Is that what you meant? Although what you describe sounds like the default, so it should already work. — Kenny Ostrom, Oct 30 '17 at 19:08
@KennyOstrom Sorry for the confusion, I meant that in `C.py`, I want to be able to use the import statements relative to the root directory (`import source.foo.lib.qux.A` without having to add to every file `basePath = os.path.abspath("../..")` and `sys.path.append(basePath)` — Nyxynyx, Oct 30 '17 at 22:51
You shouldn't: the module really shouldn't alter the path, especially not to be able to import modules from a completely different package. Perhaps the original script could do that, but ideally, it should really be the environment settings. Don't. — , Oct 30 '17 at 22:58

Piotr Dobrogost · Answer 1 · 2017-11-02T22:25:16.397

You need to make foo, lib, qux, bar and baz into packages by adding __init__.py file in each of these directories, place the source directory on the sys.path (by adding the directory to PYTHONPATH before running script for example) and use relative import syntax like from .foo.lib.qux.A import ...

UPDATE

Generally, you are expected to make a proper Python package out of your project. This is done by writing setup.cfg or setup.py file according to rules. Then you are expected to install your package. It's during installation that specific actions are taken to insure your project's directories and files (or rather packages and modules) will be importable. The most common way to install a Python package is by running pip install <package-name> or pip install --editable . in the root directory of a package.
The former copies directories and files within the package to site-packages directory specific to Python's interpreter. As this directory is put on sys.path during Python's startup packages and modules placed there are automatically importable by any Python code.
The latter is meant for a package which is under development and leads to creation of .pth file in already mentioned site-packages directory. The contents of a file is a path to package's root directory. All .pth files are read during Python's startup and paths added to sys.path list.

If you want to make packages and modules importable without installing your project/package then you can extend sys.path yourself.
You can create .pth file with the path to your project's root directory and place it in appropriate site-packages directory yourself. To find out where's site-packages directory for the Python instance you are using refer to How do I find the location of my Python site-packages directory?
Or you can set environment variable PYTHONPATH before running Python – PYTHONPATH=/path/to/the/project/root/dir /path/to/python script-from-project.py

Do you append the `source` directory to `PYTHONPATH`, for example using `export PYTHONPATH=$PYTHONPATH:C:\path\to\source\dir` in Windows? I tried using `os.environ["PATH"] += os.pathsep + pathToSourceDir` but it seems that this needs to be added to every Python file that is executed, making it really hard to change when you have many files and the source directory path changes — Nyxynyx, Oct 30 '17 at 23:38
Maybe theres a way to add the source directory path to the virtualenv settings? Similar to what PyCharm does when setting the source directory. — Nyxynyx, Oct 31 '17 at 00:10

Programmatically append to PYTHONPATH in VirtualEnv using .pth

1 Answers1