1

I am trying to improve the project structure while adding to a code base. I found a sample structure here which looks like this:

README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py

I notice in particular that requirements.txt and setup.py are on a higher level than tests/ and sample/

If I add sample/classes.py you need only write from classes import MyClass in sample/core.py to get it in there. It cannot however so easily be imported into tests/test_basic.py, does not seem like python 'looks around the corner' like that when importing.

In my case, there is also a MANIFEST.in on the same level with requirements.txt and some files which are not really python but just set things up for the platform on which this runs.

If classes.py were on the same level as requirements.txt I think it would be easily importable by everything in tests/ and in sample/ and their subdirectories, but it may need a __init__.py That doesn't feel right somehow.

So where should it go if both tests/ and sample/ need to be able to use it?

cardamom
  • 6,873
  • 11
  • 48
  • 102
  • The whole point of the tests is that they should have access to the code in `sample`; they'd just have to import `classes` under the name `sample.classes`. – jwodder Oct 09 '18 at 15:11
  • I don't think there is a framework in there yet, was reading about those recently could be a plan for the future. For now I run tests like this: `python -m unittest discover path/to/tests` – cardamom Oct 09 '18 at 15:12
  • @jwodder I just tried that but it said `ImportError: No module named 'sample'` So I tried instead to put a dot before `from .sample.classes import classes` and it said `SystemError: Parent module '' not loaded, cannot perform relative import` – cardamom Oct 09 '18 at 15:21

2 Answers2

2

Let's make it easy.

If I understand correctly, the problem is How to import simple module in test. Which means you want to use something like from simple.classes import MyClass.

That's easy, just add your root path to PYTHONPATH before executing python test/test_basic.py.

That's also what an IDE does for you when you execute tests through it.

Sraw
  • 18,892
  • 11
  • 54
  • 87
  • Do you mean before the import statements, `import sys` then `sys.path.append('/path/to/project')`? Yes, that would probably work but I just thought there was a more elegant way to do it. – cardamom Oct 09 '18 at 15:25
  • 1
    You can just manipulate your env variable: `PYTHONPATH=/path/to/project python test/test_basic.py`. – Sraw Oct 09 '18 at 15:26
  • Ok, so `classes.py will continue to live in `sample/` . I went to the linux terminal and ran `export PYTHONPATH=/path/to/project` and then `printenv | grep PYTHON` to make sure that it was there, but that doesn't seem to have been enough to make it work. Something else needs to go near the import statements but am trying to work out what.. – cardamom Oct 09 '18 at 15:41
  • It should be enough, maybe you can update the question to clarify what's your question now. – Sraw Oct 09 '18 at 15:44
  • Thanks btw, yes that solution with PYTHONPATH did work I had a typo as I was testing it. Had two terminals open one with the environment variable and one without, and proved that it worked. Learned something new. Lot of opinions about whether that is a good idea though, eg negative opinions [here](https://stackoverflow.com/a/19917658/4288043) and [here](https://stackoverflow.com/a/1894043/4288043) and a positive opinion [here](https://stackoverflow.com/a/1893631/4288043) . The question was around style and best practice, but I give up and will just do what works and is half way elegant. – cardamom Oct 10 '18 at 13:43
1

Assuming you use a Python >= 3.3, you can simply turn the test folder in a package by adding a __init__.py module in it. Then in that __init__.py (and only there) you add the path of the parent package to sys.path. That if enough for unittest discover to use it for all the modules in tests.

My one is just:

import os
import sys

sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

Then if you need to access classes.py from one of the test modules, you can just use:

from sample import classes

or to directly import MyClass:

from sample.classes import MyClass

It just works because sample is already a package, and its parent folder has been added to sys.path when python unittest has loaded the test package.

Of course, this only works in you can have your tests in a package. If for any reason it is not an option, for example because you need to run individually the test modules, then you should put the sys.path modification directly in all the test files.

Write a path_helper.py file in the tests folder:

import os
import sys

core_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if core_path not in sys.path:  # don't add it if it is already here
    sys.path.append(core_path)

You can then import it in all test files:

import path_helper
...
Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
  • Thanks is useful, I have spent many hours though and just can't get it to quite work. Also helps if you import the class inside your proposed `__init__.py` I tried changing your `'..'` to `../sample` but can only get it to work like [this](https://stackoverflow.com/a/28054579/4288043) `from __init__ import *` I believe if tests is run as a script rather than a package this fails. – cardamom Oct 10 '18 at 13:50
  • @cardamom: You are right, my solution required all tests to be run as a package. I have just added an alternative way that has not this requirement. – Serge Ballesta Oct 11 '18 at 09:46
  • Thanks, that looks quite elegant. Will try it out soon, it avoids at least putting long ugly sys.append os.path stuff up the top of the files. – cardamom Oct 11 '18 at 09:57
  • @cardamom: well it is consistently executed in all the files, but at least you write the `sys.path.append(...` only once and only a nicer `import` in all other files... – Serge Ballesta Oct 11 '18 at 10:16