How do imports work in IPython

Question

I'm a little bewildered by exactly how import statements work in IPython. I've turned up nothing through web searches.

Implicit relative imports work with Python 2, but I don't know if that's still the case with IPython for Python 3.

Relative imports using the dot syntax dont seem to work at all:

In [6]: ls 
dsp/  __init__.py  __init__.pyc  utils/

In [7]: from .utils import capture
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-e7d50007bdd1> in <module>()
----> 1 from .utils import capture

ValueError: Attempted relative import in non-package

importing modules that use the dot syntax seems impossible:

In [8]: cd utils
/home/user/workspace/mypkg/mypkg/utils

In [9]: ls
capture/  capture.py  capture.pyc  cext/  __init__.py  __init__.pyc

In [10]: from capture import Capture
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-8c31c76d052d> in <module>()
----> 1 from capture import Capture

/home/user/workspace/mypkg/mypkg/utils/capture.py in <module>()
     17 import tarfile
     18 import re
---> 19 from .. import utils
     20 from . import flprint
     21 from select import poll

ValueError: Attempted relative import in non-package

Is there some concise documentation on this somewhere?

IPython imports are just regular Python imports, but running it inside a package is problematic, because Python doesn't treat the working directory as a package. cd up a couple of levels, so you can do `from mypkg.utils.capture import Capture`, and it should behave. — Thomas K, Aug 13 '14 at 17:00
I think I see. I'm confusing current working directory with a module's location in the package hierarchy. — orodbhen, Aug 14 '14 at 22:31
Yep, `from . import blah` only works if you're in a package. It doesn't just look in the directory wherever that file is. — Thomas K, Aug 14 '14 at 22:52

orodbhen · Accepted Answer · 2018-02-10T22:08:17.213

10

The problem is I was importing the module from a lower position in the package hierarchy than is used in the module's import statement. So if I cd into the utils directory and run

from capture import Capture

then capture becomes the top level of the hierarchy. So the import statement in the capture module

from .. import utils

goes beyond the top level. Python doesn't know what ".." refers to, because modules aren't self-aware of what package they belong to. If I change back up to the mypkg directory, I get the same problem

In [13]: cd ..
/home/user/workspace/myproj/mypkg

In [14]: from utils import capture
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-c87f26b2171d> in <module>()
----> 1 from utils import capture

/home/user/workspace/myproj/mypkg/utils/capture.py in <module>()
    18 import re
    19 import zmq
---> 20 from .. import utils
    21 from . import flprint
    22 from select import poll

ValueError: Attempted relative import beyond toplevel package

In this case, utils is the top level, so

from . import flprint

will work, but

from .. import utils

won't work.

I have to move one more directory up:

In [19]: cd ..
/home/user/workspace/myproj

In [20]: from mypkg.utils import capture

In [21]: cap = capture.Capture

IPython can import packages and modules located in the current working directory, or from directories in the import path. I can add the package to the import path to be able to import from any working directory.

In [23]: import sys

In [24]: sys.path.append('/home/user/workspace/myproj')

In [25]: cd
/home/user

In [26]: from mypkg.utils import capture

You can use sys.path.append to make your module "self-aware" but, as pointed out by alpha_989, it can potentially lead to name collisions. However, it is a useful workaround when doing work inside the package hierarchy from an interactive terminal session.

edited Feb 10 '18 at 22:08

answered Aug 14 '14 at 22:59

orodbhen

2,644
3
20
29

Couple of questions: – alpha_989 Feb 09 '18 at 22:33
if you add the `sys.path` to the Python path, and if there are 2 modules with the same name but located in different directories, which ever one of these modules are found first will be used? Then you have to know the names of all the modules that are already used, so that you don't cause _name-collisions_ while writing the program? – alpha_989 Feb 09 '18 at 22:35
The hypothesis seems to be that Python doesn't know that there exists a package _utils_, when you are running the file from _utils/capture_, so you have to go a level up, so python know that _utils_ is a package. When we go up a level to the _mypkg_ directory and have a `from .. import utils` in _capture.py_, there is a ___init__.py_ file in the current working directory, which should be visible to Python. **Then why doesnt it still work, when we do a `from .. import capture` work when we run/import capture while in the _mypkg_ directory?** – alpha_989 Feb 09 '18 at 22:53
There is a _capture_folder and a _capture.py_ in the same directory. How does python know that it should load the module _capture.py_ and not the _capture_ directory? – alpha_989 Feb 09 '18 at 22:55
1

@alpha_989 First question: I believe that's correct. So use relative paths when writing package modules, but `sys.path` is useful when working inside the package hierarchy from an interactive session. Second question: Because the python interpreter doesn't care about where you are. It interprets `..` in reference to your import statement, not your position in the file hierarchy. You told it that utils is the top, so any reference to `..` would have to be two levels down from that. – orodbhen Feb 10 '18 at 21:58
1

@alpha_989 Third question: The package (i.e. folder) will have preference. See [this post](https://stackoverflow.com/questions/6049825/python-what-does-import-prefer-modules-or-packages). Great questions. Thanks for asking them. Imports are a bit complicated. – orodbhen Feb 10 '18 at 21:59
1

Thanks for pointing that post out. Not my post, but somebody else pointed this out to me. You may find this post helpful: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html – alpha_989 Feb 13 '18 at 19:07

How do imports work in IPython

1 Answers1