38

In one of my testing scripts in Python I use this pattern several times:

sys.path.insert(0, "somedir")
mod =  __import__(mymod)
sys.path.pop(0)

Is there a more concise way to temporarily modify the search path?

planetp
  • 14,248
  • 20
  • 86
  • 160
  • Note: `sys.path.pop(0)` works only if `mymod` doesn't prepend another path without removing it. And if that happened pop would remove the wrong path. – weshouman Jun 16 '21 at 10:18

4 Answers4

39

You could use a simple context manager:

import sys

class add_path():
    def __init__(self, path):
        self.path = path

    def __enter__(self):
        sys.path.insert(0, self.path)

    def __exit__(self, exc_type, exc_value, traceback):
        try:
            sys.path.remove(self.path)
        except ValueError:
            pass

Then to import a module you can do: 

with add_path('/path/to/dir'):
    mod = __import__('mymodule')

On exit from the body of the with statement sys.path will be restored to the original state. If you only use the module within that block you might also want to delete its reference from sys.modules:

del sys.modules['mymodule']
Eugene Yarmash
  • 142,882
  • 41
  • 325
  • 378
  • 1
    This is a nice solution. Is it better to remove the path as you don't know the code within the context manager's context is not going to modify sys.path itself? – demented hedgehog Mar 15 '17 at 00:32
  • Yes, it may be better in such cases. Just use `sys.path.remove(self.path)` instead of `sys.path.pop(0)`. – Eugene Yarmash Jun 20 '17 at 22:11
  • 1
    to add to @dementedhedgehog 's suggestion: you might also want to catch a `ValueError` raised by `sys.path.remove` in case the code in the context has already removed the path itself – bunji Oct 17 '18 at 14:25
20

Appending a value to sys.path only modifies it temporarily, i.e for that session only.

Permanent modifications are done by changing PYTHONPATH and the default installation directory.

So, if by temporary you meant for current session only then your approach is okay, but you can remove the pop part if somedir is not hiding any important modules that is expected to be found in in PYTHONPATH ,current directory or default installation directory.

http://docs.python.org/2/tutorial/modules.html#the-module-search-path

Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
  • I cannot think of a different way, however as already mentioned, the variable sys.path will only keep its modified value within this Python session. Normally, there is no need to remove a path from sys.path, at least I cannot think of a good reason to do so. A really permanent modification of sys.path would require that you modify the PYTHONPATH environment variable of your shell. – Johannes P Jun 20 '13 at 10:23
  • 1
    @JohannesP If "somedir" is hiding(masking) some module that is expected to be found in found in `PYTHONPATH`,etc then we can remove that entry to fix that. – Ashwini Chaudhary Jun 20 '13 at 10:29
  • 2
    I remove the directory after import to not slow down the search (there potentially can be hundreds of paths) – planetp Jun 20 '13 at 10:35
  • @planetp then it's perfectly okay. – Ashwini Chaudhary Jun 20 '13 at 10:38
6

Here is an alternative implementation of the contextmanager implementation from Eugene Yarmash (use contextlib and pathlib.Path-compatible):

import os
import sys
import contextlib
from typing import Iterator, Union

@contextlib.contextmanager
def add_sys_path(path: Union[str, os.PathLike]) -> Iterator[None]:
    """Temporarily add the given path to `sys.path`."""
    path = os.fspath(path)
    try:
        sys.path.insert(0, path)
        yield
    finally:
        sys.path.remove(path)


with add_sys_path('/path/to/dir'):
    mymodule = importlib.import_module('mymodule')
normanius
  • 8,629
  • 7
  • 53
  • 83
Conchylicultor
  • 4,631
  • 2
  • 37
  • 40
  • For someone in the future that got here but can't understand what `Union` and `Iterator` are doing: search phrases "annotation" and "type hint/check" may help. – Nuclear03020704 May 07 '21 at 14:44
  • `Union` is basically like 'one or the other', and since the function yields, it returns an iterator – TankorSmash Dec 29 '21 at 19:38
3

If you're testing uses pytest, they have a great fixture that, among other traits, handles this exact case:

The monkeypatch fixture helps you to safely set/delete an attribute, dictionary item or environment variable, or to modify sys.path for importing... All modifications will be undone after the requesting test function or fixture has finished. The raising parameter determines if a KeyError or AttributeError will be raised if the target of the set/deletion operation does not exist

In describing syspath_prepend:

Use monkeypatch.syspath_prepend to modify sys.path which will also call pkg_resources.fixup_namespace_packages and importlib.invalidate_caches().

sample use:

def test_myfunc(monkeypatch):
  with monkeypatch.context() as m:
    m.syspath_prepend('my/module/path')
    mod =  __import__(mymod)
  # Out here, the context manager expires and syspath is reset
ntjess
  • 570
  • 6
  • 10