Cannot run Python3 script from subfolder // relative import error

Question

I have a simple project structure like this:

➜  (venv:evernote) evernote_bear_project git:(master) ✗ tree | grep -v pyc
.
├── README.md
...
(snip)
...
├── manage.py
├── sample
│   ├── EDAMTest.py    <==== here is an import that won't work
│   └── enlogo.png
└── util
    ├── __init__.py
    ├── files.py        <====== This is being imported
    └── test_files.py

Now I have a relative import in sample/EDAMTest.py:

from ..util.files import * 

When I try to run python sample/EDAMTest.py from project root folder in command line, I get an error saying:

ValueError: attempted relative import beyond top-level package

I know this has been asked many times, but I still don't get it.

Since I'm running the script from the project root, in my understanding Python should be able to "know" that when I try to import from ..util.files import *, that it should go up one directory, no?

---

EDIT

Thanks for all the answers.

So what I understand from the link above is this:

I was running the module sample/EDAMTest.py directly via python sample/EDAMTest.py, and that meant

1. that the __name__ of the module was __main__
2. and now the path (sample/) was my "root" so to speak.

So now Python searches only this path and any path that's below it for modules / packages. Hence the error message attempted relative import _beyond top-level package_, so it cannot go one more level up, because it is at the root already.

Also Python cannot look one level "up", since this syntax from ..util.files import * does not go up a level in directory, but in a list of modules / packages it keeps on the "import search path" (sys.path)?

Is that correct?

Answer 1

sys.path.append() is a tweak, if your directory structure is fixed and there is nothing you can do about it.

Otherwise, you can try rearranging the folders. The easiest is moving util under sample, another option is making both of the folders psrt of a larger package.

Also import * is not ensorsed.

Answer 2

The relative import syntax is for importing other modules from the same package, not from the file system.

Depending on what you want, you could...

Fix the package so that the relative import works

Put __init__.py files in the project root and sample directory and run the script from one level up. This doesn't seem like what you want.

Tell python where to find the package

Set the PYTHONPATH environment variable so that python can find the package.

PYTHONPATH='.':$PYTHONPATH python samples/EDAMTest.py

Install util so that python can find it

Add a setup script and use it to install the util package and avoid setting PYTHONPATH.

Answer 3

"The relative import syntax is for importing other modules from the same package, not from the file system.", This is right as stated by George G.

Put __init__.py in your subfolders, which will make them package.

__init__.py can be an empty file but it is often used to perform 
setup needed for the package(import things, load things into path, etc).

However you can import File into your __init__.py to make it 
available at the package level:

 # in your __init__.py
 from util import files

 # now import File from util package
 from util import files
 of if you want to import some specific method or class, you can do 
 from util.files import some_function


Another thing to do is at the package level make util/modules 
available   with the __all__ variable. When the interpeter sees 
an  __all__ variable defined in an __init__.py it imports the 
modules listed in the __all__ variable when you do:

from package import *

__all__ is a list containing the names of modules that you want to be
imported with import * so looking at our above example again if we 

wanted to import the submodules in util the all variable in util/init.py would be:

 __all__ = ['files', 'test_files']

With the __all__ variable populated like that, when you perform

 from util import *

it would import files and test_files.