22

I have the following:

         ModuleFolder
              |
              |-->. ModuleFile.py .
              |
              '-->. TestsFolder .
                         |
                         '---> UnitTest1.py

I'm trying to import from the parent directory. In this case I am trying to run "UnitTest1.py" from the test folder, and import from the directory immediately above it (the file "ModuleFile.py").

  • I know there are plenty of answers to this already. SO Question1, SO Question2, Every Other SO Question. I just couldn't find "using ../" as a relative import rather than the explicit path.
  • I know that as of Python 2.5 they supported "relative imports" as per the documentation that mentions the use of from .. import * but I am specifically trying to do an import MyModuleName so I can be more explicit in the unittest and avoid mangling/collisions of names.

What I am doing (and it is working for me) is the following:

sys.path.append("../")

And then importing what I need from the parent directory.

  • Yes, there is an __init__.py in the parent directory,
  • No, my parent path is not part of the Python path or environment variable
  • Why don't I just add the parent path to the sys.path? Because it's relative. If I am running from /home/workspace/MyModule/unittests/ and my module is under /home/workspace/MyModule/ I assumed adding /home/workspace/MyModule/ to the path won't necessarily be true if a coworker runs this on his machine under his own directory of /home/documents/MyModule.

My Question:

Is this Python-proper? If not, what's wrong with this. Is there a better way? Or is this truly an RTFM moment where the answer is in one of the 7+ SO questions I've already looked at? (I saw those all recommending the explicit path rather than the relative path approach I took).

Other useful information:

  • Python 2.6
  • Working in Linux but could just as easily jump over to Win.
Community
  • 1
  • 1
Justin Carroll
  • 1,362
  • 1
  • 13
  • 37
  • 4
    It's better to start using your code as a `package` and importing everything as such. Relative imports are then irrelevant. – Asclepius Oct 29 '13 at 21:30

6 Answers6

29

Don't run the test from the tests folder. Run it from the root of your project, which is the module folder. You should very rarely need to muck with either sys.path or PYTHONPATH, and when you do, you're either causing bugs for other libraries down the road or making life harder on your users.

python -m TestsFolder.UnitTest1

If you use a test runner like py.test, you can just run py.test from the root of your checkout and it'll find the tests for you. (Assuming you name your tests something more like test_unit1.py. Your current naming scheme is a little unorthodox. ;))

Eevee
  • 47,412
  • 11
  • 95
  • 127
  • 2
    Is this understood among "seasoned pythonistas" in that tests are rarely run from the test directory and that tests should typically be invoked at the module level? It seems counter-intuitive that a test should be ran from the parent directory when you might just as well go to the test directory and type `python test.py` to see the results -- I mean this is done with all other non-test *.py files. – Justin Carroll Oct 29 '13 at 22:59
  • 4
    no it isn't. running an importable file with `python foo/bar.py` is a bad idea, because it adds `foo/` to `sys.path`, and that's very rarely what you want. run entry points with `-m`, and run tests with a test runner. (i very very rarely `cd` any deeper than my project root.) – Eevee Oct 29 '13 at 23:10
  • 1
    I always forget about python -m. Need to have a sticker on the wall of my cubicle or something, because forgetting it has cost me a couple solid hours of productivity multiple times. +1 – Jacklynn Oct 22 '14 at 22:16
5

It's better to insert your relative path at the begening of sys.pathlike this:

import sys
sys.path.insert(0, '../')
Laurent LAPORTE
  • 21,958
  • 6
  • 58
  • 103
  • Not sure why you got voted down. but I appreciate the effort (honestly!). Why is it better to put it at the beginning? This isn't the first time I've seen this recommendation but I've never understood why... – Justin Carroll Oct 29 '13 at 23:32
  • 1
    @JustinCarroll: You put the path at the beginning to be sure your code is called even in case of paths clashing. – Laurent LAPORTE Nov 04 '13 at 14:47
2

My suggestion is:

Don't try to be clever, do what you're supposed to do. I.e. make sure that your modules and packages are somewhere in your Python path.

The simplest way is to set the environment variable PYTHONPATH in the shell that you use to execute your scripts:

$ export PYTHONPATH=/the/directory/where/your/modules/and/packages/are
$ cd /the/directory/where/your/unit/tests/are
$ python test1.py
codeape
  • 97,830
  • 24
  • 159
  • 188
  • 7
    the problem I have with this is that when someone else pulls this down to run on their environment, they get errors (false positives) in the test because they won't have their environment variable properly set. Is that their problem and it's best to follow convention here? I'm shocked that there isn't a better solution than "each user that wants to run the unittest must first set their python environment variable else they can't run it"... am I being zealous about this? (Thanks for the quick response) – Justin Carroll Oct 29 '13 at 21:03
  • I believe it is best to follow convention. Make sure your tests fail fast. You normally place all your imports at the top of the unit test module files. The unit tests will then fail at once with an ``ImportError``. Teach your coworkers what the error means and how to fix it (i.e. ``export PYTHONPATH=...`` or ``SET PYTHONPATH=...`` in Windows). – codeape Oct 29 '13 at 21:21
  • the _simplest_ way to do this is to run `python` from the directory containing your package root. – Eevee Oct 29 '13 at 21:48
  • is there _anything_ wrong (where wrong == potentially code breaking) by adding `sys.path.append("../")`? I totally got that it's 'bad form' and against convention and there are better, more normative, alternatives to solve this problem -- and, I like the suggestions made here and by @Eevee, but now I'm interested in the actual impact of doing this so I can advise others on the pros/cons (beyond it breaking convention). – Justin Carroll Oct 29 '13 at 23:03
  • sure: where exactly is `..`? if i'm not running Python from exactly the same place you are (perhaps i've installed your library!), it could be anywhere and pull in anything. – Eevee Oct 29 '13 at 23:08
2

Make your test folder a module (by adding __init__.py).
Then you can use run python -m tests.test_name or use the following Makefile in your project home:

TEST_FILES = $(wildcard tests/test_*.py)
TESTS = $(subst .py,,$(subst /,.,$(TEST_FILES)))
all.PHONY: test
test:
    @- $(foreach TEST,$(TESTS), \
        echo === Running test: $(TEST); \
        python -m $(TEST); \
        )
Shubham Chaudhary
  • 47,722
  • 9
  • 78
  • 80
2

You can also make symlink of the module path which you want to import and then use that symlink to import. Create a symlink in the python dist-packages.

To create a symlink:

ln -s "/path/to/ModuleFolder" "/path/to/python/dist/packages/module_symlink_name"

To import module in the script:

from module_symlink_name import ModuleFile

No need to export python path or modifying sys path.

PalFS
  • 731
  • 6
  • 16
0

This is always very cofusing. Relative imports are relative to the module structure. To test relative imports, you need to import your module in your main file (from outside the module). Let me try to show a case that works:

$ python use_module.py
Hello World!

Or you can make it an executable module and run it with python -m (always from outside the module)

$ python -m module.child_a
Hello World!

Structure

test_module.py
module
├── __init__.py [can be empty]
├── child_a
   ├── __init__.py
   ├── __main__.py
│── child_b
   ├── foo.py

test_module.py

from module.child_a import message
print(message)

foo.py

phrase = "Hello World"
mark = '!'

child_a/__init__.py

from .__main__ import message

child_a/__main__.py

from .. child_b.foo import phrase, mark

message = phrase + mark

if __name__ == "__main__":
    #this block is run when executing `python -m module.etl`

    print(message)

NOTE: if you add another file in child_a say child_a/myscript.py you can still execute it with python -m module.sub_a.myscript (always from outside the module)

MarcoP
  • 1,438
  • 10
  • 17