Python: ensure os.environ and sys.path are equal: web-requests, shell, cron, celery

Question

I want to ensure that os.environ and sys.path are identical for all ways we start the Python interpreter:

• web requests via Django, and Apache mod_wsgi
• Cron jobs
• Interactive logins via ssh
• Celery jobs
• Jobs started via systemd

Is there a common way to solve this?

If yes, great: How does it look like?

If no, sad: Everybody solves this on his own. ... What is a good way to solve this?

Operating System: Linux (with systemd support)

Update

More explicit:

1. I want sys.path to be the same in web requests, cron jobs, python started from shell, ...
2. I want os.environ to be the same in web requests, cron jobs, python started from shell, ...

Update2

For systemd we use EnvironmentFile

Update3

We use virtualenv

Answer 1

You can use envdir python port (here is the original) for managing the environment variables.

If you are only concerned about Django, I suggest using envdir from your settings.py programmatically

You can update the environment programmatically (e.g.: in the wsgi file, django's manage.py, settings.py, etc.)

import envdir
import os

# print os.environ['FOO']  # would raise a KeyError

path = '../envdir/prod'
if not os.path.isdir(path):
    raise ValueError('%s is not a dir' % path)
envdir.Env(path)
print os.environ['FOO']

or you can run the your process through envdir on the command line, e.g.: envdir envs/prod/ python manage.py runserver

I suggest creating aliases for python, pip, etc. (as you don't want to overwrite the system's own python), e.g.: alias python-mycorp="envdir /abs/path/to/envs/prod/ python" (or if you prefer, write a full shell script instead of an alias).

Answer 2

This mapping is captured the first time the os module is imported, typically during Python startup as part of processing site.py. Changes to the environment made after this time are not reflected in os.environ, except for changes made by modifying os.environ directly.

They all have to use the same interpreter. If they launch by the same user, they probably are.

Answer 3

As you can see in the documentation of sys.path, it is initialized with the environment variable PYTHONPATH and then with an installation dependent default (site). So, they are intended to be different.

But, you can use the -S option during the interpreter invocation: python -S script.py in order to skip some site specific configuration hook. Nevertheless, you will still have the standard library stuff in your sys.path.

If you really really want os.path['PYTHONPATH'] == sys.path, you should do it explicitly, as the documentation says:

A program is free to modify this list for its own purposes

The standard places to put those kind of specific manipulations are:

• A sitecustomize module, typically created by a system administrator in the site-packages directory, which can do arbitrary configurations.
• A usercustomize module, which intention is the same as sitecustomize but only executed if ENABLE_USER_SITE is true.
• Customization to the sys.path directly from the script. I.e: sys.path = os.env['PYTHONPATH'].

Answer 4

I'm going to assume you meant os.environ['PYTHONPATH'] == sys.path , because otherwise I can't understand the question. Anyway, the solution would be to use virtualenvs.

1. Setup a virtualenv
2. Edit the /bin/activate and add entry PYTHONPATH=your-sys-path.
3. Make sure your mod_wsgi, celery, cron jobs and shell login(bash_login?) all activate the virtualenv when they are started and use the virtualenv/bin/python for execution.

Done.