3

I recently decided to upgrade to python 3, and start converting some of my scripts. I encountered a problem in a script that uses a module named io - in python 2, this is perfectly fine, however in python 3, io is a standard module for files. I found this old question about the same kind of problem, however this appears to be in reference to python 2. I have the opposite problem - given two files, main.py and io.py in the top level package, import io in main.py will import the standard io module, not the local one. from __future__ import absolute_imports didn't help, and from . import io and related attempts fail as expected (which I have never understood - python really doesn't know where the top level package is?). Renaming is obviously a solution, but if possible I'd like to avoid it. Is there some python 3 standard way of resolving module name conflicts?

Community
  • 1
  • 1
Monchoman45
  • 517
  • 1
  • 7
  • 17
  • There's a similar question and answer here - http://stackoverflow.com/questions/1224741/python-import-with-name-conflicts – Calvin Cheng Nov 04 '12 at 08:20
  • Yes, I linked that: "I found [this old question](http://stackoverflow.com/questions/1224741/python-import-with-name-conflicts) about the same kind of problem, however this appears to be in reference to python 2." – Monchoman45 Nov 04 '12 at 08:23
  • In Python3, absolute imports are default. So:- `from .base import BaseThing` The leading . says 'import Base from module directory'; in other words, `.base` maps to `./base.py`. – Calvin Cheng Nov 04 '12 at 08:27
  • Or `from .io import *` for your case. – Calvin Cheng Nov 04 '12 at 08:28
  • `from .io import *` throws `SystemError: Parent module '' not loaded, cannot perform relative import` because both modules are in the top level package. Related forms like `from . import io` throw the same error. – Monchoman45 Nov 04 '12 at 08:32
  • Please include the full traceback in the question. – Lev Levitsky Nov 04 '12 at 08:55
  • @LevLevitsky: That's the whole traceback. `main.py` is invoked from terminal, runs `from .io import *`, throws error. No function calls or anything. – Monchoman45 Nov 04 '12 at 09:05
  • Do you have `__init.py__` in the package dir? If not, can you try creating an empty file with that name? – Lev Levitsky Nov 04 '12 at 09:07
  • I tried with and without `__init__.py`, same error. At least in python 2, this was default behavior - I experienced similar problems in unrelated scripts and could never figure out why (from a technical standpoint) python doesn't allow relative imports in the top level package. – Monchoman45 Nov 04 '12 at 09:17

1 Answers1

0

Here's my answer:-

My directory structure:-

calvin$ tree /Users/calvin/work/learn3/
/Users/calvin/work/learn3/
└── myspecialpackage
    ├── __init__.py
    ├── __init__.pyc
    ├── io.py
    ├── io.pyc
    └── main.py

__init__.py is an empty file.

io.py is your custom module which conflicts with python3's io module.

main.py contains this bunch of example code:-

import os
import sys

# These two lines are not needed you are installing the `myspecialpackage` via pip/pypi and as setup.py script places "myspecialpackage" and all its contents in your python site-packages, which is already in PYTHONPATH.
our_package_root = os.path.dirname(os.path.realpath(__file__))
sys.path.append(our_package_root)

from myspecialpackage import io
print(io.__file__)

And the imported io module will be the one in your io.py and not python3's module.

As a bonus, using this methodology will allow us to have your custom io.py as well as python3's io module (if you so desire having your cake and eat it ;-)). You can deconflict the use of the namespace io like this:-

from myspecialpackage import io as my_special_io
print(my_special_io.__file__)

import io
print(io.__file__)

Running main.py will then give you:-

In [3]: run myspecialpackage/main.py
/Users/calvin/work/learn3/myspecialpackage
./myspecialpackage/io.py
/Users/calvin/.virtualenvs/learn3/bin/../lib/python3.3/io.py

Take note of the comment I made above regarding

our_package_root = os.path.dirname(os.path.realpath(__file__))
sys.path.append(our_package_root)
Calvin Cheng
  • 35,640
  • 39
  • 116
  • 167
  • This doesn't work either - seems to be related to the other answer (that the author deleted?) which suggested `sys.path.insert(1, '/path/to/package')`. Is it possible that my installation is misconfigured? I also noticed that you're using 2.7, and I have 3.3. Does that change anything significantly? – Monchoman45 Nov 04 '12 at 09:27
  • That's not important. I was using iPython shell to do some print outs which still uses python2. Python normal shell works fine. – Calvin Cheng Nov 04 '12 at 09:29
  • Are you sure you are following exactly the directory structure I described above? – Calvin Cheng Nov 04 '12 at 09:30
  • The only difference is that `__init__.py` has no corresponding `.pyc`. That shouldn't change anything, though. – Monchoman45 Nov 04 '12 at 09:34
  • The important part is to import your special io module using the `from myspecialpackage import io` syntax. This guarantees that that `io` you are importing comes from `myspecialpackage` and not the system-wide/python3's io.py. – Calvin Cheng Nov 04 '12 at 09:36
  • Ah - I didn't pick that up the first time. With that it throws `ImportError: bad magic number in 'a_package': b'\x03\xf3\r\n'`. I have no idea what that's supposed to be telling me. – Monchoman45 Nov 04 '12 at 09:43
  • 1
    Changing sys.path is unrelated to OP's issue that is solved by putting the modules inside myspecialpackage and using absolute imports. btw, `learn3` should be in sys.path, not `myspecialpackage` directory. `python -mmyspecialpackage.main` should work without modifying sys.path inside main.py. – jfs Nov 04 '12 at 09:44
  • @Monchoman45: "bad magic number" means that you're trying to use a pyc-file generated by a different Python version. Note: Python 3 adds the version in the file name so there shouldn't be any conflict (in addition Python 3 puts pyc-files to `__pycache__` directory). Just delete pyc-files; they will be regenerated when necessary – jfs Nov 04 '12 at 09:51
  • That worked, however now it throws `ImportError: No module named 'myspecialpackage'`. This I would assume is because it's the top level package? – Monchoman45 Nov 04 '12 at 16:45
  • It means that your `myspecialpackage` is still not in `PYTHONPATH` – Calvin Cheng Nov 04 '12 at 16:48
  • `myspecialpackage` is the first entry in `sys.path` - however per @J.F.Sebastian above, should it actually be the directory above `myspecialpackage`? – Monchoman45 Nov 04 '12 at 17:22