Multiple relative imports in python 2.7 packages

Question

I understand there are many SO questions on relative imports. I will document the extent I have tried the solutions therein, but I am still unable to solve my problem.

I have the following directory structure. It's not mine by design but I'm game for modifying things as necessary (forked repo).

exp
  main_Exp.py
  kaffe
    __init__.py
    tensorflow
      __init__.py
      network_shape.py
  ResNet
    __init__.py    (*)
    ThreeDMM_shape.py

To run the model in this repo, I am to use

python main_Exp.py input_file.csv

Inside main_Exp.py:

sys.path.append('./kaffe')
sys.path.append('./ResNet')
from ThreeDMM_shape import ResNet_101 as resnet101_shape
from ThreeDMM_expr import ResNet_101 as resnet101_expr

Inside ResNet/ThreeDMM_shape.py:

sys.path.append('/home/usc/Desktop/Research/FG18/ExpNet_Code_Release/kaffe/tensorflow')
from network_shape import Network_Shape

Ok, so obviously I need to change this hard-coded absolute path. I'd like to do it the right way and not use my own specific path that I happened to install these files to.

So I try

from ..kaffee.tensorflow import Network_Shape

>>> ValueError: Attempted relative import in non-package

(1) I added __init__.py file in the ResNet folder (shown with the (*)) (2) I tried running the file as a module: python -m main_Exp input_file.csv (3) I also tried adding __init__.py to the top level folder (exp), though I believe doing so is nonsense. (4) Given that the first import was happening using the kaffe path that was appended to sys.path, I tried changing import to from .tensorflow.network_shape import Network_Shape

Same error after all steps.

So I'm not understanding the rules around relative imports and how to reference files in a sane way. I would really appreciate a pointer that helps me understand how to do this, and how to think about such imports in general!

Mr_and_Mrs_D · Accepted Answer · 2019-03-14T21:46:13.777

1

exp is indeed not a package and won't be made to a package even if you add an init file to it, cause that won't magically add it to the paths python looks for packages. If you do add the __init__.py you can then run as python -m exp.main_Exp input_file.csv (from ../exp). This would make python recognize exp as a package and kaffe/ResNet as subpackages. You would need to change imports to from ResNet.ThreeDMM_shape import ResNet_101 as resnet101_shape etc.

Edit in response to comment: Running from the parent dir using the m switch is the recommended way of running the script see for instance https://stackoverflow.com/a/23540051/281545 (that's for python 3 however it should still apply). If you want to avoid it (it would break hardcoded relative paths for one) you should add exp to sys path (once maybe is enough) then change the imports to absolute ones as in:

# main_Exp.py
sys.path.append(os.path.abspath(os.path.dirname(__file__))) # the exp folder
from ResNet.ThreeDMM_shape import ResNet_101 as resnet101_shape
from ResNet.ThreeDMM_expr import ResNet_101 as resnet101_expr

# ResNet/ThreeDMM_shape.py
from kaffee.tensorflow import Network_Shape

edited Mar 14 '19 at 21:46

answered Mar 14 '19 at 16:10

Mr_and_Mrs_D

32,208
39
178
361

Ok this helps; so as written, `kaffe` is a package, `ResNet` is a package, and if they both just happen to live in the non-package folder `exp` there's no python-related way that they know about each other outside of the package search path, is that right? I hesitate to simply add a whole bunch of folders to the path, but is this simply how it's done? – Andrew Schwartz Mar 14 '19 at 19:20
1

I am not sure whether or not this is the best way to structure it all. This way you drop the `sys.append` calls - they are taken care of by python with the `-m` there. Alternatively you would append just `exp` to `sys.path` and then use absolute imports (`from kaffe.tensorflow ...`). By adding just `exp` python will scan it for packages and discover `kaffe` and `ResNet`. This way it adds`exp.kaffe` and `exp.ResNet`. – Mr_and_Mrs_D Mar 14 '19 at 20:30
That smelled bad to me too, which is why I didn't want to add more. The code is not mine originally; ideally I want to fix the imports in a way that can be merged back to the upstream repo. That's why I'm also wary of requiring users to run it with `-m`, since that's a breaking change to the current API, so I'm trying to understand python imports more broadly to fix it soundly. – Andrew Schwartz Mar 14 '19 at 20:57

Multiple relative imports in python 2.7 packages

1 Answers1