0

I have the following directory structure:

some-tools-dir/
    base_utils.py
    other_utils.py
    some-tool.py
    some-other-tool.py

some-other-tools-dir/
    basetools -> symlink to ../some-tools-dir
    yet-another-tool.py

In other_utils.py, I have:

import base_utils

Now, in yet-another-tool.py, I want to do:

import basetools.other_utils

That doesn't work, because Python does not recognize basetools as a Python package. So I add an empty basetools/__init__.py. Now, in other_utils, I get the exception:

    import base_utils
ImportError: No module named base_utils

So I change that line to:

from . import base_utils

And yet-another-tool.py works now.

However, some-tool.py does not work anymore. It imports other_utils, and there I get the exception:

    from . import base_utils
ValueError: Attempted relative import in non-package

Now, I can add this hack/workaround to some-tools-dir/*-tool.py:

import os, sys
__package__ = os.path.basename(os.path.dirname(os.path.abspath(__file__)))
sys.path += [os.path.dirname(os.path.dirname(os.path.abspath(__file__)))]
__import__(__package__)

And in addition, make all local imports relative in those files.

That solves the problem, I guess. However, it looks somewhat very ugly and I have to modify sys.path. I tried several variations of this hack, however, I want to support multiple Python versions if possible, so using the module importlib becomes complicated, esp. because I have Python 3.2, and I don't like using module imp because it's deprecated. Also, it only seems to become more complicated.

Is there something I'm missing? This all looks ugly and too complicated for a use-case which doesn't seem to be too uncommon (for me). Is there a cleaner/simpler version of my hack?

A restriction I'm willing to make is to only support Python >=3.2, if that simplifies anything.

Lev Levitsky
  • 63,701
  • 20
  • 147
  • 175
Albert
  • 65,406
  • 61
  • 242
  • 386

2 Answers2

1

(Note that this answer was built by piecing together information from this answer and this question, so go up-vote them if you like it)

This looks a bit less hacky, and at least works with Python 2.7+:

if __name__ == "__main__" and __package__ is None:
    import sys, os.path as path
    sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))

from some_tools_dir import other_utils

I think the main reason you're finding this difficult is because it's actually unusual to have executable scripts inside of a python package. Guido van Rossum actualy calls it an "antipattern". Normally your executable lives above the root directory of the package, and then can simply use:

from some_tools_dir import other_utils

Without any fuss.

Or, if you want to execute a script that lives in the package, you actually call it as part of the package (again, from the parent dir of the package):

python -m some_tools_dir.other_utils
Community
  • 1
  • 1
dano
  • 91,354
  • 19
  • 222
  • 219
  • 1
    Note that I used directory names with "-" by intention in my question, because I wanted to have a general solution. Your solution assumes that it is a valid package name (`some_tools_dir`). --- However, I decided to restructure my files and follow Guidos advice. Also, the `python -m` is not a real solution because I want to be able to have the scripts executable and I don't see how I can accomplish that via hashbang. – Albert Jul 19 '14 at 11:10
0

Are you be able to add the top root path into PYTHONPATH ?

If so, you can then add 

__init__.py

file into some-tools-dir (and/or some-other-tools-dir)

Then from other_utils.py you do 

from some-tools-dir import base_utils

And in yet-another-tool.py you do 

from some-tools-dir import other_utils

You can then remove the symlink, and have proper namespacing.

Wan B.
  • 18,367
  • 4
  • 54
  • 71
  • I already did that. That is what I wrote in my question. Also, your code does not work because module names can not contain `"-"`. – Albert Jul 19 '14 at 11:07