67

This should be the easiest problem on earth, but even after extensive searching and tinkering, I'm still in deep trouble with finding a "correct" way to lay a directory structure and manage to run pytest etc correctly.

Let's say my I have a program called apple.

|- README.md
|- apple
|   |-- __init__.py
|   |-- apple.py
| - tests
|   |-- test_everything.py

The apple.py contains some functions, for examples sake let's call one eat(). And the test_everything.py file contains some tests like assert eat()=="foobar". So good so easy, but then the fun begins:

  • What about the __init__.py in the apple directory... correct? Empty or what should be inside?
  • Is it best practice to call py.test from the root directory? Or py.test tests?
  • So many projects have a __init__.py in their test directory, but that's explicitly said to be wrong in the py.test documentation. So why god why
  • What comes at the top of the test_everything.py file: an import apple or from apple import *? or something else entirely
  • Do you call the functions then by eat() or apple.eat()?
  • Some even recommend manipulating os.path.dirname in python

This should be easy, but I've seen every combination of the above, not even speaking about tox and the myriad of other tools. Yet with the slightest error, you get thrown some ImportError: No module named 'apple' or some other funky error.

What is the "right" way? The advice and the existing code on github etc follows extremely different conventions. For a medium-experienced coder, this should be much easier.

Basti
  • 2,228
  • 1
  • 19
  • 25

2 Answers2

37
  1. What about the __init__.py in the apple directory... correct? Empty or what should be inside?

Yes, correct. Most frequently empty. If you put foo = 42 in it you can later do from apple import foo while you'll need to do from apple.apple import foo if you put it in apple.py. While it might seem convenient you should use it sparingly.

  1. Is it best practice to call py.test from the root directory? Or py.test tests?

py.test should be able to find your tests regardless, but see below..

  1. So many projects have a __init__.py in their test directory, but that's explicitly said to be wrong in the py.test documentation. So why god why

So you can import a file in tests that provide common test functionality. In py.test that might be better achieved by creating fixtures in a file called tests/conftest.py.

  1. What comes at the top of the test_everything.py file: an import apple or from apple import *? or something else entirely

from apple import apple

  1. Do you call the functions then by eat() or apple.eat()?

apple.eat()

  1. Some even recommend manipulating os.path.dirname in python

That seems very fragile. I would suggest either

(a) set the environment variable PYTHONPATH to point to the folder where README.md is, or better

(b) create a setup.py file (at the same level as your README.md file), here's a minimal one:

from setuptools import setup
setup(name='apple', packages=['apple'])

Run the file like so:

python setup.py develop

now apple is globally available and you should never see a no module named apple problem again, i.e. you can run py.test from the root folder or the tests folder.

You can read more about setup.py in the Python Packaging User Guide at https://packaging.python.org/

Drdilyor
  • 1,250
  • 1
  • 12
  • 30
thebjorn
  • 26,297
  • 11
  • 96
  • 138
  • 2
    I've [created a repository](https://github.com/Seanny123/derpland) showing how to accomplish the task using the `setup.py` method. – Seanny123 May 19 '16 at 15:56
  • `from apple import apple` doesn't work inside `test_everything.py`: `ModuleNotFoundError: No module named 'apple'` – endolith Jun 13 '17 at 02:19
  • @endolith it does, although probably not if cwd = apple or you didn't run setup.py develop, etc. This is an accepted answer from 2014 with a score of 7, if you can't get it to work there's a good chance it's a problem at your end - and you're much more likely to get help by asking a new question than by leaving a comment lacking context/details... – thebjorn Jun 13 '17 at 08:45
5

I have arranged an example that works here.

I think naming both package and module apple confusing, perhaps that was one source of confusion. The only non-obvious part IMO is that you must set PYTHONPATH to the current directory if you're not using a proper setup.py distutils package to install your apple package.

Bruno Oliveira
  • 13,694
  • 5
  • 43
  • 41
  • It is confusing indeed. My assumption was that the first 'apple' from `from apple import apple` was referring to the folder name. Therefore the package name. The second 'apple' refers to the filename. So, the name of the module. Now, for a project to really be set up, a mandatory setup.py file must be correctly configured and executed. A great deal of information must be known beforehand, I suppose? How many packages will I need, how will they be named. Or is there a common workflow using setup.py? Like hitting `python setup.py` once in a while to introduce the project to new files? – atripes Aug 04 '17 at 12:52
  • 1
    @atripes you generally only need to run `python setup.py develop` or better `pip install -e .` (note the dot, and you must be in the directory with the setup.py file). The only time you need to re-run it is if you have other local projects that rely on it (i.e. have something like `-e ../apple` in their requirements.txt file) and you change the install_requires parameter to setup. It really quite simple ;-) – thebjorn Jan 25 '22 at 15:51