5

I'm trying to import models from a folder in the parent directory. Im using sys.path.append(). My project structure:

-Project

  • folder1
    • file1.py
    • ...
  • folder2
    • file2.py
    • ...

In file1.py file:

sys.path.append('../Project')
from Project.folder2 import file2

I then get a:

ModuleNotFoundError: No module named Project

I know there are other ways but this seems like the simplest. I'm not sure if I need to put the absolute path to the Project folder, but I'm hoping not since I'll be running this Project on different computers (diff abs path).

Alex Waygood
  • 6,304
  • 3
  • 24
  • 46
chadlei
  • 179
  • 1
  • 11
  • 2
    do you have a ```__init__.py``` file in your Project directory? – ewokx Sep 16 '21 at 00:46
  • Shouldn't this be `sys.path.append('..')`, since you're using `from Project....`? – joanis Sep 16 '21 at 01:03
  • 1
    @ewong I do not. I read somewhere that __init__ files are no longer needed with python3, but whats your thoughts? – chadlei Sep 16 '21 at 16:54
  • @joanis whats the exact append statement? ".." isnt valid – chadlei Sep 16 '21 at 17:24
  • I would have expected `sys.path.append('..')` to add the parent directory of your current working directory. But maybe Python doesn't allow that? Anyway, my point is that if `xyz/Project` is on sys.path, Python would look for module `Project` in `xyz/Project/Project/` instead of directly in `xyz/Project/`. – joanis Sep 16 '21 at 18:55
  • I just tested `sys.path.append('..')` with a simpler setup, and it worked for me. – joanis Sep 16 '21 at 18:59
  • I just read Niel's answer, and this is a bad idea anyway, because the path is going to be relative the where your process is running, not relative to where your file is located. – joanis Sep 16 '21 at 19:02
  • The way I work with things like this is to actually do an in-place pip install: create a `setup.py` and run `pip install -e .` at the root of your project, so that it's actually installed and visible to Python from anywhere. – joanis Sep 16 '21 at 19:04
  • @joanis interesting! i went with niels way of just adding the abs path. with your way, is setup.py like a requirements.txt file? and you just put all the apps you want in there? – chadlei Sep 16 '21 at 22:01
  • The setup.py file can have the contents of the requiments.txt file, or in my case, read the requirements.txt file. But it contains everything else needed for `pip install -e .` to know what to do. And it will be required if you publish your project to PyPI at some point. Have a look at https://stackoverflow.com/q/4740473/3216427 for some examples. – joanis Sep 17 '21 at 18:38

3 Answers3

6

2 errors in your code:

  1. The Project directory is not just 1-level up. From the point of view of file1.py, it is actually 2 levels up. See this:
$ cd ..
(venv) nponcian 1$ tree
.
└── Project
    ├── folder1
    │   └── file1.py
    └── folder2
        └── file2.py
(venv) nponcian 1$ cd Project/folder1/
(venv) nponcian folder1$ ls ..
folder1  folder2
(venv) nponcian folder1$ ls ../..
Project
  1. Even if the above works, adding a relative path as a string would literally append that string as its raw value. So if you add a print(sys.path), it would display something like this:
['/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '../Project', '.']
  • It literally added '../Project', so if python started searching for the target modules in folder2, it wouldn't still find them because how would it know where exactly is '../Project' relative to.

Solution

What you need to add is the absolute path. If your problem is it might change, it is fine because we don't need a fixed absolute path. We can get the absolute path through the location of the current file being executed e.g. file1.py and then extracting the parent directory needed. Thus, this will work regardless if the absolute paths change because the way we are getting it is always relative to file1.py. Try this:

Project/folder1/file1.py

from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent.parent.parent))  # 1. <.parent> contains this file1.py 2. <.parent.parent> contains folder1 3. <.parent.parent.parent> contains Project

... the rest of the file
Niel Godfrey Pablo Ponciano
  • 9,822
  • 1
  • 17
  • 30
  • this makes sense and is a great solution to my problem. it's still not working unfortunately but I feel close. when I print out my path after adding your suggestion above, my path is still /Users/chad.lei/Downloads/Project1/folder1. what I should be seeing is /Users/chad.lei/Downloads/ right? Also, when I print out Path(__file__).parent I get a '.' returned. – chadlei Sep 16 '21 at 16:53
  • i also tried using resolve on Path(__file__) first. still getting . for some reason – chadlei Sep 16 '21 at 17:55
  • Just like how you originally planned with doing `../../..`, you just have to keep on chaining `.parent.parent.parent.` until you reach the target parent directory relative to the executed file. I hope you already resolved it :) – Niel Godfrey Pablo Ponciano Sep 17 '21 at 00:00
  • 1
    Hey, that's a really cool tool, `tree`! – joanis Sep 17 '21 at 20:57
  • @NielGodfreyPonciano can you help with another similar issue? I have a file (File A) in the parent directory that imports File B from a folder in the same directory as A. File B imports a file (File C) from the same directory it's in. If I run File B directly, it runs fine. But If I run File A, File B gives me a ModuleNotFoundError: No module named 'File C'. What can I do here? – chadlei Sep 23 '21 at 18:18
  • @chadlei Perhaps you performed an implicit relative import when you imported C into B that's why it didn't work when you changed your current path to the outer directory. If ever it is indeed implicit relative import, try changing it to explicit `from .C import something` (notice the `.` in the beginning of `C`). Of course it could also be absolute path `from parentx.parenty.C import something` – Niel Godfrey Pablo Ponciano Sep 25 '21 at 01:28
  • I'll give that a try thanks ! – chadlei Sep 27 '21 at 23:46
2

TL;DR

You can solve this by creating a setup.py file for your project and then running pip install -e ., instead of modifying sys.path.

Motivation

This answer might seem to come from left field for this question, but I'm putting it here because OP showed interest in this solution, and I think it's generally a preferable solution to mucking around with sys.path.

Details

The solution I tend to prefer for my projects is to create an actual setup.py for them, as if I was going to publish them to PyPI, and then run pip install -e . to have them actually installed in my (possibly virtual) python environment.

Here's a minimalist setup.py similar to one I used before:

from setuptools import setup

setup(
    name="project",
    version="0.0.1",
    python_requires=">=3.6",
    author="Some cool dude",
    long_description="Some cool project",
    install_requires=["dep1", "dep2"],
)

In another project, I read my requirements.txt file in my setup.py:

from setuptools import setup

with open("requirements.txt") as f:
    requirements = f.read().splitlines()

setup(
    name="project",
    version="0.0.1",
    python_requires=">=3.6",
    author="Some cool dude",
    long_description="Some cool project",
    install_requires=requirements,
)

With either solution, the setup.py file is sibling to my project directory (I use lowercase for my project names, always), both typically being at the root of my Git repo, or under a src subdirectory.

cd to the directory where setyp.py and project are, and run

pip install -e .

and now from anywhere, you can import project or its submodules and Python will find them.

Suggested reading

Lots more details can be found about setup.py files at setup.py examples?

joanis
  • 10,635
  • 14
  • 30
  • 40
  • can you help with another similar issue? I have a file (File A) in the parent directory that imports File B from a folder in the same directory as A. File B imports a file (File C) from the same directory it's in. If I run File B directly, it runs fine. But If I run File A, File B gives me a ModuleNotFoundError: No module named 'File C'. What can I do here? @joanis – chadlei Sep 23 '21 at 18:18
  • This might be worth asking as its own question, with a pointer to this one for context. But, let me try to understand first... A, B, and C are in the parent dir of Project? So, they're not in a dir your python path, right? Are they in the directory where you are running? – joanis Sep 23 '21 at 18:26
  • A and Folder Z that contains B and C are in the parent directory. I'm running A which is in the parent directory. They are not in my python path – chadlei Sep 23 '21 at 18:29
  • Generally, I use import statements that fully qualify where to import from, all the time. So if Z.B imports Z.C, I say `import Z.C` even in Z.B, rather than rely on any kind of localness of the files. – joanis Sep 23 '21 at 18:29
  • If `import Z.B` worked in A, then I would expect `import Z.C` to work in Z.B in that same execution environment. – joanis Sep 23 '21 at 18:31
  • oh ok specifying the file within B worked! thanks again – chadlei Sep 23 '21 at 18:47
  • No problem! Glad I could help. – joanis Sep 23 '21 at 19:13
1

I encountered the same issue in Windows OS. In addition to possible level injection as a parent, the problem with sys.path.append is that it creates a windows path which Python is not searching for some reason despite it being added to the sys.path. Instead try to use

ROOT_PATH = pathlib.Path(__file__).parents[1]

 sys.path.append(os.path.join(ROOT_PATH, ''))

It will add file's grandparent to sys.path in a proper way.

Elchin Guseynov
  • 409
  • 4
  • 8