0

"What are the PEP-compatible ways to import a local package even from a parent folder with the least preassumptions" could be the long title. I am looking for a solution to make it possible to use a local package. My requirements:

  1. it must be PEP-compatible, i.e. at least the most commonly accepted PEP must support it
  2. OS, terminal, user and environment independent, i.e. the solution must not rely on any specific circumstances

Among these solutions, I am looking for the one that

  • is the simplest to implement
  • the scope of the modification is flexible (doesn't need to be executed every time, but is not permanent and system-wide)
  • leaves the source code in the simplest form, i.e. the solution doesn't need a lot of extra lines in the code

Solutions

These are the ways I found on SO and other places so far. None of them suit my needs, and as it was told 8 years ago, every solution will have their cons. Yet I hope in 2021 we can find a solution that meets at least my 2 requirements.

Use conda

SO 1

Use conda environments and issue conda to include a directory in the python lookup path by conda develop <path> where <path> can be either relative to pwd or absolute. Conda then creates a conda.pth file in the site-packages folder of the environment, and python will be provided the content of that file, therefore will be available in the sys.path

Drawback: it requires conda-build, but it is not always present.

Use pip

SO 2a, SO 2b, SO 3

Use pip to install a package locally. Create a setup.py, configure it, cd into where it is, then install the package locally in editable mode with pip install -e .

Drawback: PEP 517 doesn’t support editable installs as mentioned on setuptools.

direct hack of sys.path

SO 2c

Add paths to sys.path directly within the python script, e.g.:

import os
import sys

sys.path.insert("path")

import mymodule

Here path can be absolute or relative, and the latter can be OS independent e.g. by using os.path.sep.

Drawback: it is not PEP-compatible, because not all import statements are in the top of the file. autopep8 will rearrange your code into a wrong order, however, there are workarounds. One can say that it is a good example where PEP can be neglected but I'd like to believe imports are basic enough to apply PEP without exception.

modify pythonpath

Modify your system environemnt path or pythonpath by adding the necessary directory to it.

Drawbacks: it is at least OS dependent and too global, or just the opposite, one needs to execute it every time.

use relative imports

From within the script, include the module using relative import

from ..somewhere import mymodule

This doesn't work if this module is the main, you'll get a

ImportError: attempted relative import with no known parent package

furthermore, PEP 8 says: "Relative imports for intra-package imports are highly discouraged." Guido thinks it is ok.

Related questions

This is not a duplicate of the general questions on how to import modules beyond top-level, because those questions already have accepted answers. SO 2, SO 3, SO 4, SO 5 SO 6, not even python

DanielTuzes
  • 2,494
  • 24
  • 40
  • 2
    What do you mean by "PEP compatible"? – juanpa.arrivillaga May 14 '21 at 17:15
  • A solution is PEP compatible if at least the most commonly accepted PEPs support it by explicitly suggesting them or not having any rules against it. You can have more understanding of this by reading the "Use pip" and the "direct hack of `sys.path`" sections which describe non-PEP compatible practices. – DanielTuzes May 14 '21 at 23:25
  • 1
    Yes, that is what isn't clear. For example, the PIP solution seems the most normal to me. But you mention that editable installs are incompatible with that PEP, but I don't see how that is the case. Furthermore, why do you *have* to use `-e` anyway? – juanpa.arrivillaga May 15 '21 at 00:07
  • [np8's answer](https://stackoverflow.com/a/50194143/1837006) and [pytest's good integration practices](https://docs.pytest.org/en/6.2.x/goodpractices.html) contained the `-e` switch. [Setuptools says](https://setuptools.readthedocs.io/en/latest/setuptools.html#setup-cfg-only-projects) it is not supported but I didn't dig deep into PEP to figure out if there is a way meet PEPs. I'd love if you could show it is in line with PEP to use pip. – DanielTuzes May 15 '21 at 20:56
  • On the Conda side: Note that `conda develop` is a symlink-based solution, just as `pip -e`. If you don't wish to write a recipe.yaml (i.e., use `conda-build`), then one can also still use `pip -e` from the Conda environment (or [in a YAML](https://stackoverflow.com/a/63387456/570918)). – merv May 16 '21 at 01:58
  • Also, I'm very torn about this question: it's excellently documented, but the problem still seems underdetermined (i.e., open to opinion). To me you've essentially asked the (clear and distinct) question "*How **can** I do this?*", but then proceeded to give many possible answers in the question already, and end up asking SO users to argue over the predefined answers. Were this clearly constrained to only Conda environments, then I'd suggest this is a duplicate of the question I linked above. – merv May 16 '21 at 02:17
  • 1
    @merv I came up with solutions I found to show the efforts on my side. Any further solution or explanation is welcomed, the list I provided is not exhaustive and means no restriction. E.g. if one could explain why `pip -e` is a good practice and how it is just a misunderstanding or oversimplification that it is discouraged, that would be a perfect solution. – DanielTuzes May 16 '21 at 13:51

1 Answers1

0

As PEP8 states in the very beginning:

A Foolish Consistency is the Hobgoblin of Little Minds.

Think about that and either use editable installs with pip or develop installs with conda,
but don't consider one of the other hacks, since they serve a different purpose.

Peter
  • 10,959
  • 2
  • 30
  • 47
  • 1
    Why would PEP explicitly discourage the use of the best practice to import a module? If it is not a big deal to use editable install then they wouldn't introduce such recommendation imho. So far I used the pip solution if conda wasn't available because these methods were the ones where linters don't produce warnings. – DanielTuzes May 14 '21 at 23:29
  • 2
    If a package doesn't support editable installs that doesn't mean they are discouraged.They are working just fine. On the other hand you are breaking the rules yourself: Your post ist highly opinionated and shouldn't have been put on SO in the first place. – Peter May 15 '21 at 06:46
  • I believe this is not only my desire to find a way to include a package that doesn't go against the recommendations therefore my aspects are not opinion-based. The way I'd like to order the proper solutions is arbitrary, you are right, it has some personal taste, but so far no proper solution is presented. – DanielTuzes May 15 '21 at 10:55