0

I want to have my tests in a separate folder from my package code, such that from the top level directory of my project I can run python sample/run.py or python tests/test_run.py, and have both of them resolve all the imports properly.

My directory structure looks like this:

sample/
   __init__.py
   helper.py
   run.py
tests/
   context.py
   test_run.py

I know there are supposedly many ways to achieve this, as discussed here: Python imports for tests using nose - what is best practice for imports of modules above current package

However, when I try to run python tests/test_run.py, I get a ModuleNotFoundError for 'helper', because 'sample/run.py' imports 'sample/helper.py'.

In particular, I am trying to follow the convention (suggested in the Hitchhiker's Guide to Python) of explicitly modifying the path using:

import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

As a result, I have a blank sample/__init__.py, along with the following code files.

sample/run.py:

from helper import helper_fn
def run():
    helper_fn(5)
    return 'foo'
if __name__ == '__main__':
    run()

sample/helper.py:

def helper_fn(N):
    print(list(range(N)))

tests/context.py:

import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import sample

tests/test_run.py:

from context import sample
from sample import run

assert run.run() == 'foo'

So I have two questions:

  1. Why is Python unable to find the 'helper' module?
  2. How do I fix things so that I can run both sample/run.py and tests/test_run.py from the top-level directory?
CDspace
  • 2,639
  • 18
  • 30
  • 36
camall3n
  • 45
  • 5

1 Answers1

2

Edited:

To make both sample/run.py and tests/test_run.py work, you should add the path of sample directory into python path. So, your tests/context.py should be 

import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../sample')))


import sample

This change will let Python know the path of helper module.

sample/run.py should be:

  from .helper import helper_fn
  def run():
     helper_fn(5)
     return 'foo'
  if __name__ == '__main__':
     run()

Implicit relative imports within packages are not available in Python 3. Please check below:

The import system has been updated to fully implement the second phase of PEP 302. There is no longer any implicit import machinery - the full import system is exposed through sys.meta_path. In addition, native namespace package support has been implemented (see PEP 420). link

This documentation might be helpful to understand Intra-Package-References.

YoungChoi
  • 324
  • 4
  • 14
  • This fixes the import error when running `tests/test_run.py`, but it leads to `ModuleNotFoundError: No module named '__main__.helper'; '__main__' is not a package` when running `sample/run.py`. – camall3n Jul 03 '18 at 03:46
  • The linked documentation says it's not possible to use relative imports in a module intended for use as the main module, so I tried abstracting the `if __name__ == 'main': run()` into a new file called `sample/main.py` (along with `from run import run`). When using the relative import suggested above (`from .helper ...`), it works for `tests/test_run.py` and fails for `sample/main.py`. With the original import (`from helper ...`), it works for `sample/main.py` but not `tests/test_run.py`. – camall3n Jul 03 '18 at 04:03
  • The relative import suggested above *does* work if you run `python -m sample.run`, as mentioned here: https://stackoverflow.com/a/23542795/. I still find this pretty unsatisfying though, since now there are two ways you need to invoke python, depending on whether it's a test script or a project script. Is there any way where I can still call both scripts as specified at the beginning of the question? – camall3n Jul 03 '18 at 04:20
  • @camall3n I edited my solution. This change will make both codes work and can answer your first question. – YoungChoi Jul 03 '18 at 04:20
  • This still seems kind of ugly, but I'll admit that it does what I asked for. I'm accepting this answer unless someone can think of a cleaner solution. – camall3n Jul 03 '18 at 04:36