19

I have a question about imports. The question might seem a bit contrived, but its purpose is to explore the limitations of using absolute imports for all imports in a package. PEP8 highly discourages relative imports (edit: and the Google Python Style Guide says never to use them).

Say you are given two large-ish packages that both have the same name and that both use absolute imports in accordance with PEP8:

    /pkg1
        mod1.py (contains an absolute import: 'import pkg1.mod2')
        mod2.py
        ...

    /pkg1
        mod1.py (contains an absolute import: 'import pkg1.mod3')
        mod3.py
        ...

Also say you are working on a Python project in which you'd like to use both packages. This could come up, say, if you want to use two versions of the same package in a project.

Is there a way to incorporate both packages into your project hierarchy so that you can freely use modules from both packages throughout your project?

For the solution it's acceptable to do things like use import aliases and modify sys.path temporarily. But it's not okay to change the contents of either package directory.

cjerdonek
  • 5,814
  • 2
  • 32
  • 26
  • 1
    Readability is important, but not as important as correctness. – Ignacio Vazquez-Abrams Feb 21 '11 at 05:57
  • The easiest thing to do is just rename one of the packages base name. Python doesn't really support versioned packages. – Keith Feb 21 '11 at 06:11
  • @Keith: is there a way to "rename one of the package's base name" without altering one of the package directories itself? Also, the packages aren't necessarily different versions. That was just an example. – cjerdonek Feb 21 '11 at 06:18
  • @Keith: It does, with a little help from `pkg_resources` in setuptools. – Ignacio Vazquez-Abrams Feb 21 '11 at 06:18
  • @ignacio setuptools manages versions, but I don't think it lets you use two versions at the same time. – Keith Feb 21 '11 at 06:23
  • @chris Well, if you don't need different simulataneous versions you can use "package namespaces" feature of setuptools (if you own both packages). If you only own one you should probably just rename yours (the directory name is the package name). – Keith Feb 21 '11 at 06:25
  • 1
    PEP8 is a great starting point, but not the last word; after all, "A Foolish Consistency is the Hobgoblin of Little Minds". – Thomas Edleson Feb 21 '11 at 06:32

4 Answers4

22

The short answer is no, Python doesn't accept two packages with the same name. (There are things called "namespace packages" that let a single package be implemented over multiple directories but they require the packages involved to be set up to cooperate with each other).

The way PEP 8 discourages explicit relative imports is one of its more questionable pieces of advice, precisely because it makes it harder to rename a package to avoid naming conflicts. If the two packages used relative imports, you could just rename one of them or nest it inside another package and be done with it.

import aliases won't help you here, because it is the name that ends up in sys.modules that matters, and that uses the name of the module as imported, rather than the name that ends up being bound in the importing module.

If you wanted to get really exotic, you could write your own importer (see PEP 302 and the 3.x importlib documentation). If you decide to go that far, you can do pretty much anything you want.

ncoghlan
  • 40,168
  • 10
  • 71
  • 80
  • 1
    Thanks. Do you mean [PEP302](http://www.python.org/dev/peps/pep-0302/) instead of [PEP303](http://www.python.org/dev/peps/pep-0303/)? – cjerdonek Feb 21 '11 at 07:36
  • The package name does not need to be the import name. One example is [PIL](https://pypi.org/project/PIL/) and the drop-in replacement [Pillow](https://pypi.org/project/Pillow/) ([source](https://github.com/python-pillow/Pillow/blob/master/setup.py#L898)) – Martin Thoma Nov 01 '20 at 08:09
  • What happens in that case? – Martin Thoma Nov 01 '20 at 08:09
3

My initial tests (in Python 2.6 & 3.1) suggest the following may work:

import sys, re

import foo as foo1
for k in sys.modules:
    if re.match(r'foo(\.|$)', k):
        newk = k.replace('foo', 'foo1', 1)
        sys.modules[newk] = sys.modules[k]
        # The following may or may not be a good idea
        #sys.modules[newk].__name__ = newk
        del sys.modules[k]

sys.path.insert(0, './python')
import foo as foo2
for k in sys.modules:
    if re.match(r'foo(\.|$)', k):
        newk = k.replace('foo', 'foo2', 1)
        sys.modules[newk] = sys.modules[k]
        # The following may or may not be a good idea
        #sys.modules[newk].__name__ = newk
        del sys.modules[k]

However, I only tested this against very simple packages and only tried it as a curiosity. One problem is it probably breaks reload. Python isn't really designed to handle multiple packages with the same top-level name.

At this point, I'm tentatively going to say that it's not possible in the general case, though it's possible under certain limited circumstances but it's very brittle.

outis
  • 75,655
  • 22
  • 151
  • 221
  • 1
    That will only work if you import all submodules before renaming anything in sys.modules and if the code doesn't contain any deferred imports (i.e. importing from inside a function). `__name__` in all the modules will be wrong as well, as will any `__module__` attributes. As your last comment noted, Python really isn't set up to support this scenario. – ncoghlan Feb 21 '11 at 07:23
  • I'm almost prepared to say "impossible in the general case", but haven't looked enough at module internals to be sure. – outis Feb 21 '11 at 07:27
  • 2
    In response to this StackOverflow question, on IRC the author of [Exocet](http://washort.twistedmatrix.com/2011/01/introducing-exocet.html) told me that his package was written to address issues like this. – cjerdonek Feb 21 '11 at 08:00
0

Actually, you should use namespaces (packages) to separate properly what modules you want to end up using. In your above code.

/pkg1
 mod1 - can just import mod2
 mod2.py
 __init__.py

/pkg2
 mod1 - can just import mod2
 mod2.py
 __init__.py

And at rest of the places you should do import pkg1.mod1 or import pkg2.mod1 as desirable.

Senthil Kumaran
  • 54,681
  • 14
  • 94
  • 131
0

No, but you can use keyword "as" and rename the package name as you wish. EG:

from tableaudocumentapi import Workbook as table
vikraman arut
  • 57
  • 4