6

I really hope this is a simple case of me miss-understanding the complex Python2 import mechanisms. I have the following setup:

$> ls -ltr pypackage1 
total 3
-rw-r--r-- 1 pelson pelson   0 Aug 17 19:20 io.py
-rw-r--r-- 1 pelson pelson   0 Aug 17 19:20 __init__.py
-rw-r--r-- 1 pelson pelson  57 Aug 17 19:22 code.py
$> cat pypackage1/code.py 
from __future__ import absolute_import

import zipfile

i.e. I have nothing but a stub package with an empty __init__.py and io.py, and a 2 lines code.py file.

I can import pypackage1:

$> python -c "import pypackage1.code"

But I cannot run the code.py file:

$> python pypackage1/code.py
Traceback (most recent call last):
  File "pypackage1/code.py", line 3, in <module>
    import zipfile
  File "python2.7/zipfile.py", line 462, in <module>
    class ZipExtFile(io.BufferedIOBase):
AttributeError: 'module' object has no attribute 'BufferedIOBase'

Clearly the problem has to do with the zipfile module picking up my relative io module over the builtin io module, but I thought my from __future__ import absolute_import would have fixed that.

Thanks in advance for any help,

Seb D.
  • 5,046
  • 1
  • 28
  • 36
pelson
  • 21,252
  • 4
  • 92
  • 99
  • Is it not possible to rename the `io.py` to e.g `__io.py` – Kimvais Aug 17 '12 at 19:18
  • Why does `zipfile.py` appear to be in a local directory? – ecatmur Aug 17 '12 at 20:25
  • I tried to replicate this. If I run it from *outside* the package directory, it worked without raising any errors. From within the package directory, it raises the same `BufferedIOBase` error that you mention. This is a weird problem, but is it really going to be an issue for you in practice? (particularly if you use the normal Python package structure) – Jeff Tratner Aug 17 '12 at 20:28
  • @JeffTratner that is how (relative) imports have always worked in Python. The current directory is automatically part of `sys.path`. – dsh Aug 17 '12 at 20:48
  • @Kimvais: Of course I *could*, but thats not a very nice solution (and I won't learn something new that way). :-) – pelson Aug 17 '12 at 22:22
  • @ecatmur: That was just me copying and pasting as much as was necessary. I don't have any fruity python setup going on just in case you are wondering. – pelson Aug 17 '12 at 22:23
  • @JeffTratner: The reason I am asking this question is because it was a problem for me in practice. I have a "tests.py" module which was runnable, but was getting these namespacing issues. Sounds like the proper solution is to not directly invoke subpackage modules, instead wrapping them outside of the package. – pelson Aug 17 '12 at 22:26
  • @pelson, I guess I usually avoid this issue by putting the tests and the package files each in separate directories and then running the test from the base directory. – Jeff Tratner Aug 18 '12 at 03:29
  • Actually, _not_ using overlapping names with standard library packages and builtins _is_ a nice solution, in my opinion. – Kimvais Aug 18 '12 at 05:28
  • .. And, since we're talking about collisions with stdlib names, how about ***not*** using `code` as a module name? Afterall, there *is* a `code` module in standard lib ;) – MestreLion Nov 16 '12 at 16:00
  • The point is, its my namespace (I'm not talking about defining top level packages of these names), so I can put whatever I want in there. Be it ``code``, ``io`` or anything else. @Bakuriu's answer was the correct solution. – pelson Nov 16 '12 at 16:11

3 Answers3

7

That's the correct behaviour. If you want to fix the error simply do not run from inside the package.

When you run a script which is inside the package, python wont interpret that directory as a package, thus adding the working directory to the PYTHONPATH. That's why the io module imported by the zipfile module is your io module and not the one inside the standard library.

I'd recommend to create a simple launcher script outside your package (or in a bin/scripts folder), and launch that. This script can simply contain something like:

from pypackage1 import code

code.main()

An alternative to this is to tell the python interpreter that the file that you want to execute is part of a module. You can do this using the -m command line option. In your case you would have to do:

python -m pypackage1.code

Note that the argument of -m should be the module name, not the file name.

Bakuriu
  • 98,325
  • 22
  • 197
  • 231
  • "When you run a script which is inside the package, python wont interpret that directory as a package, thus adding the working directory to the [`sys.path`]." However, by default, it **will** add the current working directory to the front of `sys.path`. If this folder contains its own `io.py`, therefore, the problem re-emerges. This can be addressed using command-line switches and/or the `PYTHONSAFEPATH` environment variable. – Karl Knechtel Jan 16 '23 at 09:54
1

One solution would be to put from __future__ import absolute_import in the zipfile.py module. Although your module is using absolute import, the zipfile module is not.

Another option is to not run from your package directory. You probably shouldn't be running the interpreter from within the package directory.

dsh
  • 12,037
  • 3
  • 33
  • 51
  • Yes, I think this would work. Sadly, zipfile is the python stock zipfile module and I have no desire to modify the code (nor do I actually have the permissions to either). One of the reasons I am asking this question is because I am curious as to why the standard python library doesn't use ``from __future__ import absolute_import``. – pelson Aug 17 '12 at 22:18
  • I agree that modifying the standard library isn't recommended practice. I suspect that the standard library doesn't use the future-import because changing it is unnecessary: it already works the way it has always worked. If the future imports were added, then they should subsequently be removed when the new behavior becomes standard. – dsh Aug 18 '12 at 14:09
  • Aside from the issue with modifying standard library files, this **wouldn't fix the problem**. The problem occurs because, with absolute import **already enabled** by the `__future__` import, the attempt to `import io` searches `sys.path`, and finds the project's code before other standard library code. – Karl Knechtel Jan 16 '23 at 09:53
1

File structure:

test.py
mylib/__init__.py
mylib/__collections.py
mylib/collections.py
mylib/mymod.py

This solution allows for:

  • test.py to invoke both __builtin__.collections and mylib.collections
  • mymod.py to invoke the above, both when it runs as part of the library and when it is run standalone (e.g. for test code)

In test.py:

from collections import deque
from mylib.collections import mydict

In mylib/__init__.py:

from __future__ import absolute_import
from . import collections
from . import mymod

In mylib/__collections.py:

class MyDict (dict):
    pass

In mylib/collections.py:

from __collections import *

In mylib/mymod.py:

from __future__ import absolute_import
from collections import deque
try:
    # Module running as part of mylib
    from .collections import MyDict
except ValueError:
    # Module running independently
    from __collections import MyDict

The above works with Python >=2.5. Python 3 doesn't need the lines 'from __future__ import absolute_import'.

crusaderky
  • 2,552
  • 3
  • 20
  • 28