python - importing parent dir issues

Question

I created with virtualenv a Foo project and I've some problem with importing the File module in test_file.py

This is my project directory

Foo/
├── app
│   ├── __init__.py
│   ├── testfile.py
│   ├── tests
│   │   ├── __init__.py
│   │   └── test_gram.py
│   ├── util
│   │   ├── File.py
│   │   ├── Gram.py
│   │   ├── __init__.py
└── NOTES

test_gram.py:

from app.util.File import loadDataFromPickle

listofdict = loadDataFromPickle(".....")
i = 0
for item in listofdict[:50]:
    print(item)

If I run test_gram, I am getting a ImportError:

Traceback (most recent call last):
  File "test_gram.py", line 1, in <module>
    from app.util.File import loadDataFromPickle
ImportError: No module named 'app

What am I doing wrong? Do I need to change the virtualenv-path to Foo/app instead of Foo?

Answer 1

If test_gram.py is in the tests folder, then the import line should be:

from ..util.File import loadDataFromPickle$

---

Another option is to use the imp module, which is usually suggested instead of appending the sys.path (source here, including Python 3 version)

import imp

foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()

Answer 2

from app.util.File import loadDataFromPickle

is an absolute import, meaning

1. go to sys.path
2. from any of the paths listed there
3. find the first one that has an app module.
4. Then import the util module from that app module.
5. ...

The tricky bit is the sys.path, which is documented as

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

So if you run Foo/app/tests/test_gram.py, sys.path starts with .../Foo/app/tests/. There's no app module anywhere below that directory, so you cannot import app using an absolute import (unless there's an app somewhere else on some sys.path).

As suggested in the comments and other answers, it is good practice to use relative imports in such situations:

from ..util.File import loadDataFromPickle

Relative imports are relative to the current package/module, not to the directories in sys.path.

Edit:

However, relative imports will not work when running the script directly from the command line, as python will complain that the '' module has not been imported (Parent module '' not loaded, cannot perform relative importSystemError: Parent module ''). Well, that's because the parent modules (tests and '') are not loaded when running the script directly and the importer rightly assumes they should be.

One trick is to run the test script as a module:

python -m app.tests.test_gram.py

This will most likely require some changes to the test script, at least having

if __name__ == '__main__':
    [...]

inside the script. See also Relative imports in Python 3 for more details.

As a suggestion, you might want to convert the test script to using unittest, anyway.

Answer 3

You are in test_gram.py which is in a child folder of app. Thus, when you try to import app, it looks for app in tests, which it isn't there:

import sys
sys.path.append('/Foo/app/util')

from File import loadDataFromPickle

Answer 4

In this case I just needed to run the test (Foo/app/tests/test_gram.py) from toplevel directory and not from the test directory like I did.