0

Like many others, I've had trouble with tests returning the error "ImportError: No module named {module name}". This is commonly because tests are in a separate directory from the code being tested. Fixes include making sure __init__ statements are present in the folders, having the tests in the same folder as the tested modules, or tricks to modify the python path. An example of one of the many many questions of this type is: PATH issue with pytest 'ImportError: No module named YadaYadaYada'

I'm reaching the conclusion that something is very messed up with the PYTHONPATH on my system, because:

  • tests are now not working even if placed alongside the modules they test

  • test scripts that are in the same folder as the modules being tested can actually import and use the modules, as long as they are run as a python executable and not being run by a test (e.g. pytest)!

  • the OS path fix demonstrated by Kenneth Reitz's Hitchhiker's Guide to Python, and using his samplemod package cloned from GitHub, also doesn't work on my system!

To demonstrate the first two bullet points: here is the directory for my project:

pydnmr
├── CHANGELOG.txt
├── LICENSE.txt
├── README.rst
├── docs
│   └── placeholder.txt
├── pydnmr
│   ├── __init__.py
│   ├── dnmrmath.py
│   ├── dnmrplot.py
│   ├── main.py
│   ├── plottools.py
│   ├── test_dnmrmath.py
│   └── test_plot.py
├── requirements.txt
├── setup.py
└── tests  # unused
    ├── __init__.py
    ├── context.py
    └── test_plot.py

I initially tried using Reitz's method in the tests folder, to no avail, so I gave up and moved the tests to the same directory as the code to be tested (dnmrplot.py etc). The test files, if run from the command line as a python script (python test_plot.py) import a module and run just fine, but if run as a test (pytest test_plot.py) they get the import error.

Here is the code for test_plot.py:

import pytest
# noinspection PyUnresolvedReferences
import dnmrplot


def test_dnmrplot_2spin_type():

    WINDNMR_DEFAULT = (165.00, 135.00, 1.50, 0.50, 0.50, 0.5000)
    x, y = dnmrplot.dnmrplot_2spin(*WINDNMR_DEFAULT)
    print('x ', type(x), "y", type(y))
    assert type(x) == "<class 'numpy.ndarray'>"
    assert type(y) == "<class 'numpy.ndarray'>"


if __name__ == "__main__":
    import plottools as pt

    WINDNMR_DEFAULTS = (165.00, 135.00, 1.50, 0.50, 0.50, 50.00)
    spectrum = dnmrplot.dnmrplot_2spin(*WINDNMR_DEFAULTS)
    pt.popplot(*spectrum)

Both the test, and code executed when run as __main__, require the same dnmrplot module import. Running the code as a python executable works, but pytest gives the import error.

As for Reitz's workaround: he adds a context.py file in the tests directory (also adopted by me in my tests directory shown above):

import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import sample

However, his included test test_basic.py doesn't actually test the import. I amended it so that it does:

from .context import sample

import unittest


class BasicTestSuite(unittest.TestCase):
    """Basic test cases."""

    def test_absolute_truth_and_meaning(self):
        assert True

    # I Added the following to actually require accessing 'sample':

    def test_sample_hmm(self):
        response = sample.hmm()
        assert response == 'hmmm...'

if __name__ == '__main__':
    # unittest.main()  #Reitz' code
    print(sample.hmm())  # my code

and when run either as a python script, or as a test, it fails to import sample:

    def test_sample_hmm(self):
        response = sample.hmm()
>       assert response == 'hmmm...'
E       AssertionError: assert None == 'hmmm...'

test_basic.py:18: AssertionError
----------------------------- Captured stdout call -----------------------------
hmmm...
====================== 1 failed, 1 passed in 0.06 seconds ======================
$ python test_basic.py
Traceback (most recent call last):
  File "test_basic.py", line 3, in <module>
    from .context import sample
SystemError: Parent module '' not loaded, cannot perform relative import
$ cd ..
$ python tests/test_basic.py
Traceback (most recent call last):
  File "tests/test_basic.py", line 3, in <module>
    from .context import sample
SystemError: Parent module '' not loaded, cannot perform relative import

I've tried running tests via the command line, both from within and above test files. I've also tried running the tests within my IDE PyCharm. I've tried using PyCharm to set the upper pydnmr folder as a sources root. I've fiddled with the Reitz workaround. I've tried cloning my project into a completely different location on my hard drive. At this point my hypothesis is that, between using an Anaconda install on my Mac, and using PyCharm, that there's something messed up with PYTHONPATH on my machine, and that I need an exorcist.

Using Python 3.4, on OS X , with an Anaconda install and PyCharm.

Any help with figuring out the source of my import problems would be greatly appreciated.

Community
  • 1
  • 1
Geoffrey Sametz
  • 618
  • 1
  • 6
  • 17
  • I've discovered something: when regular imports are used, the python scripts run as programs, but a) tests fail, and b) the imports are flagged with a red warning in PyCharm as unresolved references. BUT if relative (dot) imports are used, tests work, and PyCharm finds no error with them, but then the python programs won't run! e.g using `from .dnmrplot import dnmrplot_2spin` in test_plot.py and changing the references accordingly causes tests (command line or in PyCharm) to pass, but `python test_plot.py` gives `SystemError: Parent module '' not loaded, cannot perform relative import`. – Geoffrey Sametz Mar 25 '17 at 04:23

1 Answers1

0

The root of the problem appears to be with whether the module is used as an import vs. as a script. This appears to address the behavior:

Relative imports in Python 3

Community
  • 1
  • 1
Geoffrey Sametz
  • 618
  • 1
  • 6
  • 17