ImportError shows up with py.test, but not when running the app

Question

Unlike in this question: Importing modules from a sibling directory for use with py.test I can import something from my app, but there's an import error (looking like a circular dependency) that raises from 'inside' myapp while running the test and not when running myapp alone:

$ python3 myapp/myapp.py
Some dummy string (correct output)

But:

$ python3 -m pytest                                                                               
================================================================= test session starts =================================================================
platform linux -- Python 3.4.3, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/nico/temp/projects_structures/test04/myapp, inifile: 
plugins: cov-2.2.1
collected 0 items / 1 errors 

======================================================================= ERRORS ========================================================================
________________________________________________________ ERROR collecting tests/test_things.py ________________________________________________________
tests/test_things.py:4: in <module>
    from myapp.lib.core.base import do_things
myapp/lib/core/base.py:1: in <module>
    from lib import something
E   ImportError: No module named 'lib'
=============================================================== 1 error in 0.05 seconds ===============================================================

As you can see, the problem is not the import statement from the test file. It's raised from 'inside' myapp.

Here is the complete structure:

.
└── myapp
    ├── myapp
    │   ├── __init__.py
    │   ├── lib
    │   │   ├── core
    │   │   │   ├── base.py
    │   │   │   └── __init__.py
    │   │   └── __init__.py
    │   └── myapp.py
    └── tests
        └── test_things.py

myapp.py contains:

#!/usr/bin/env python3

from lib.core import base

base.do_things()

lib/__init__.py contains:

something = "Some dummy string (correct output)"

base.py contains:

from lib import something

def do_things():
    print(something)
    return True

and test_things contains:

import unittest
import sys
sys.path.insert(0, '..')
from myapp.lib.core.base import do_things

class DoThingsTestCase(unittest.TestCase):

    def test_do_things(self):
        self.assertTrue(do_things())

if __name__ == '__main__':
    unittest.main()

And $PYTHONPATH seems correctly set (so this: Py.test No module named * doesn't answer my problem). (Or if this is not correct, how can I correct it?)

$ echo $PYTHONPATH
/home/nico/temp/projects_structures/test04/myapp/myapp

I can't answer what the problem here is exactly, but messing with `sys.path` and `PYTHONPATH` is always a recipe for trouble. I'd recommend just installing your project in a [virtualenv](https://virtualenv.pypa.io/en/stable/) or using [tox](https://tox.readthedocs.io/en/latest/) and using fully qualified imports like `from myapp.lib import something`. — The Compiler, Jun 01 '16 at 07:17
@TheCompiler thanks for these hints. I didn't precise it because it was already a long post, but usually `PYTHONPATH` is empty (problem remains exactly the same). I don't like to mess with `sys.path` neither, but I have seen somewhere there's no other choices for using `py.text` with python3. So I will check if virtualenv and/or tox do help to solve the case (though I would expect them not to be necessary for that). — zezollo, Jun 01 '16 at 07:21
Where did you read that? I'm (and I'm sure countless others are) using pytest just fine with python3 without any `sys.path` shenanigans. — The Compiler, Jun 01 '16 at 12:39
Can you try this and let me know the output ? `$ export PYTHONPATH= ` and from myapp root `py.test tests/test_things.py` — Sanju, Jun 01 '16 at 14:31
@Sanju same output, unfortunately. I've tried setting to `/home/nico/temp/projects_structures/test04/myapp/myapp` and to `/home/nico/temp/projects_structures/test04/myapp`. The script works when it runs alone, but the test fails with the very same message. It looks like there's a kind of "shift" between tests and myapp. If I put `from myapp.lib import something` in `base.py`, I get the contrary: myapp does not work (ImportError), but the tests finish sucessfully! — zezollo, Jun 01 '16 at 15:11
@Sanju, OK, I should have mistyped something, as I retried `$ export PYTHONPATH=/home/nico/temp/projects_structures/test04/myapp/myapp` and it seems to have solved the case now. You could add this as an answer. — zezollo, Jun 01 '16 at 16:15
@TheCompiler I'm 100% sure I've seen that in SO (because I haven't figured out this on my own); I have been struggling to find it again, in vain, I give up. — zezollo, Jun 01 '16 at 16:26

score 3 · Accepted Answer · edited Jun 02 '16 at 05:18

3

Setting the PYTHONPATH should do the trick.

$ export PYTHONPATH=<ABSOLUTE PATH TO TOPMOST myapp dir>

(in this example, it is the path to myapp/myapp; and ensure you've exported PYTHONPATH, not only set it).

and from myapp run either

$ py.test

or

$ python3 -m pytest

edited Jun 02 '16 at 05:18

zezollo

4,606
5
28
59

answered Jun 02 '16 at 03:51

Sanju

1,974
1
18
33

zezollo · Answer 2 · 2016-06-05T08:51:41.680

Another way of doing this is to put a conftest.py file into the top level of myapp, or in myapp/tests. Like this:

$ pwd
/home/nico/temp/projects_structures/test04/myapp/tests
$ touch conftest.py
$ cd ..
$ py.test
================================================================= test session starts =================================================================
platform linux2 -- Python 2.7.6, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/nico/temp/projects_structures/test04/myapp, inifile: 
collected 1 items 

tests/test_things.py .

============================================================== 1 passed in 0.05 seconds ===============================================================
$

(and PYTHONPATH is empty:

$ echo $PYTHONPATH

$

)

This way, py.test automatically adds myapp to PYTHONPATH. This avoids to forget exporting PYTHONPATH and will make the tests of myapp easier for other devs (who neither will need to solve this problem).

ImportError shows up with py.test, but not when running the app

2 Answers2

Linked