Relative imports with unittest in Python

Question

I am trying to use Python unittest and relative imports, and I can't seem to figure it out. I know there are a lot of related questions, but none of them have helped so far. Sorry if this is repetitive, but I would really appreciate any help. I was trying to use the syntax from PEP 328 http://www.python.org/dev/peps/pep-0328/ but I must have something wrong.

My directory structure is:

project/
    __init__.py
    main_program.py
    lib/
        __init__.py
        lib_a
        lib_b
    tests/
        __init__.py
        test_a
        test_b

I run my tests using:

python -m unittest test_module1 test_module2

test_a needs to import both lib/lib_a and main_program. This is the code from test_a I am trying to use for the import:

from ..lib import lib_a as lib
from ...project import main_program

both raise this error:

ValueError: Attempted relative import in non-package

All of my init.py files are currently empty.

Any specific advice would be greatly appreciated!!

Edit:

This may be the answer: Python Packages? I'm still verifying if this will work.

Edit II:

To clarify, at this point I have attempted to run my test file in 3 different ways:

project/tests $ python -m unittest test_a
project/tests $ python -m test_a
project/tests $ ./test_a

All three fail with the same error as above. When I use the same three syntaxes but in the project directory, I get this error:

ValueError: Attempted relative import beyond toplevel package

Thanks again.

"Note that while that last case [...] is legal, it is certainly discouraged ("insane" was the word Guido used)." - PEP328 — John Mee, Jan 31 '14 at 22:24
Try invoking it as a package... http://stackoverflow.com/a/11536794/75033 — John Mee, Jan 31 '14 at 22:28
possible duplicate of [Attempted relative import in non-package even with \_\_init\_\_.py](http://stackoverflow.com/questions/11536764/attempted-relative-import-in-non-package-even-with-init-py) — John Mee, Jan 31 '14 at 22:30
I need to run it through unittest, so I don't think that will work. When I run this: 'python -m tests.test_a' I get this error: 'ValueError: Attempted relative import beyond toplevel package'. — J Jones, Feb 01 '14 at 05:18
I think you can chain `-m`'s? And check on what relative imports are relative to? The pythonpath? The current directory? What? I just take guido's view and rarely, if ever, use them. If your tests are testing the lib module then put them into the lib with the routines they are testing. Only tests that are testing the project belong at the top level there. — John Mee, Feb 01 '14 at 07:15
Chaining '-m's doesn't appear to work with unittest. I just want a way to have my tests in a separate folder than my library. I'm not attached to using relative imports, but I do think that having these separated will just be nicer structurally. Is that just a bad idea? Assuming that tests/ is only testing stuff in lib/, is there a good way to keep those directories separate? I _could_ structure it so that project/ contains tests/ which contains lib/, but then I'm importing my lib files to main_program through tests/, which just seems silly. Thanks for your comments, I appreciate it! — J Jones, Feb 01 '14 at 18:02
I recommend laying out your project structure as described in http://blog.habnab.it/blog/2013/07/21/python-packages-and-you/ — Brave Sir Robin, Feb 04 '14 at 18:53
@martinjak, thanks, but this isn't working for me either. I am assuming that I must be missing something to make this a proper 'package', but I really don't know what I'm missing. From the python interpreter in project/, these both work: import lib, from lib import lib.lib_a. However, this doesn't work: import lib.lib_a. Do I need to be doing all of this from the directory above project/? Or something else? This is my first Python package, and I'm having a really hard time finding documentation to tell me what I'm doing wrong. — J Jones, Feb 06 '14 at 01:07

score 27 · Accepted Answer · answered Mar 03 '14 at 14:48

27

In my experience it is easiest if your project root is not a package, like so:

project/
  test.py
  run.py
  package/
    __init__.py
    main_program.py
    lib/
      __init__.py
      lib_a
      lib_b
    tests/
      __init__.py
      test_a
      test_b

However, as of python 3.2 , the unittest module provides the -t option, which lets you set the top level directory, so you could do (from package/):

python -m unittest discover -t ..

More details at the unittest docs.

answered Mar 03 '14 at 14:48

kai

1,970
2
22
30

3

Thanks! This did help. I also benefitted greatly from the module section in Mark Lutz's 'Learning Python'. I realized that I need to make sure that imports work correctly from the directory in which the module will actually be called, not the directory in which the module lives. So, now I'm calling my test files from project/, and the import in test_a reads: 'import lib.lib_a' and 'import spd'. That, in combination with the simple way to run unittests from a higher level directory, enabled me to fix my code. – J Jones Mar 07 '14 at 23:48
What does test.py contain? – gsanta Jan 24 '17 at 19:09
In this case it'd be an executable file that runs the tests (something like `unittest.main()`, setting up your `sys.path` or whatever is required - it's not essential for something simple. – kai Jan 26 '17 at 10:55
1

What does ´test_a´ and ´test_b´ contain? And how do they import the contents of ´lib_a´ and ´lib_b´ for testing? – Tim Skov Jacobsen Dec 09 '18 at 13:06
Like @JJones mentioned, Mark Lutz's 'Learning Python' explains the module import process very well. Highly recommended read. – NFern Oct 05 '19 at 23:35

score 6 · Answer 2 · edited Apr 06 '20 at 05:51

6

I run with the same problem and kai's answer solved it. I just want to complement his answer with the content of test.py (as @gsanta asked). I've only tested it on Python 2.7:

from packages.tests import test_a, test_b
import unittest

# for test_a
unittest.main(test_a, exit=False)

# for test_b
unittest.main(test_b)

then you can just

../project $ python test.py

edited Apr 06 '20 at 05:51

cymbaz

25
5

answered Oct 26 '17 at 18:49

Rodrigo E. Principe

1,281
16
26

I would upvote this if the structure of test_a would also be present – rufreakde Jan 16 '20 at 14:33

score 2 · Answer 3 · answered Jan 23 '20 at 00:51

2

In a layout where tests and package are at sibling level:

/project
   /tests
   /package

One way is to start within the package dir, use -s to discover only in tests, and use -t to set the top level:

../package $ python3 -m unittest discover -s ../tests -t ..

answered Jan 23 '20 at 00:51

Eric Jarvi

181
1
7

And how do the test scripts under `/tests` import from `/package`? Is it something like `import package.foo`? – AntonK Feb 17 '23 at 10:52

Relative imports with unittest in Python

3 Answers3

Linked