3

Suppose mine.py wants to import moduleA and moduleB but moduleA and moduleB each try to import a module called "moduleC". These are two distinct modules that both happen to be named "moduleC". When mine.py is run, depending on sys.path either moduleA or moduleB gets the correct "moduleC", the other gets a surprise, and chaos ensues.

If moduleA and moduleB were written by different authors, neither of which is us, it is preferable to not modify those modules. Is there any solution available to the author of mine.py that does not modify moduleA or moduleB?

The following questions ask how to solve this issue when you are the author of moduleA or moduleB.

Importing from builtin library when module with same name exists

How to access a standard-library module in Python when there is a local module with the same name?

My specific case

I want to run a program called PyMOL under the Python debugger, pdb. PyMOL unfortunately has a "cmd.py" that it imports and that conflicts with the usual cmd which pdb imports.

The relevant parts of the PyMOL install look like this:

pymol/
    __init__.py
    cmd.py

PyMOL is run by executing __init__.py. This file then imports cmd as from pymol import cmd.

Working from what BrenBarn pointed out, so far I can get pdb to successfully import the correct cmd by temporarily removing the pymol directory from the front of sys.path. After that when PyMOL tries to import its cmd it crashes. Somehow I need to remove Python's cmd from the import module search before PyMOL imports but after pdb imports.

Minimal example

$ ls
pymol/
$ ls pymol/
__init__.py  cmd.py

init.py

# insert some code into __init__.py directly

import sys
pymol_path = sys.path[0]
sys.path[0] = ""
import pdb
sys.path[0] = pymol_path

from pymol import cmd

# test a sandwich of calls that require each "cmd" modules
pdb.set_trace()    
cmd.foo()    
pdb.set_trace()
cmd.foo()

print "done!"

# original PyMOL __init__.py code would follow

cmd.py

def foo():
    print("cmd.foo()")

Try it

$ PYTHONPATH= python ./pymol/__init__.py
> /Users/khouli/scr/pymol_scr/pymol/__init__.py(11)<module>()
-> cmd.foo()
(Pdb) continue
cmd.foo()
> /Users/khouli/scr/pymol_scr/pymol/__init__.py(13)<module>()
-> cmd.foo()
(Pdb) continue
cmd.foo()
done!

Edit: The method given above now seems to work but as BrenBarn's answer states, there likely is no solution that leaves all third party code unmodified as the question originally asked for. This is due to quirks in PyMOL.

Community
  • 1
  • 1
Praxeolitic
  • 22,455
  • 16
  • 75
  • 126
  • Are you saying that moduleA, moduleB and both moduleC are all independently-installed modules (i.e., it's not that either moduleA or moduleB is trying to import a moduleC from within a subpackage of itself, or something)? – BrenBarn Nov 20 '14 at 03:33
  • I had in mind that "moduleC" is a subpackage of both moduleA and moduleB. Could you explain why it makes a difference? – Praxeolitic Nov 20 '14 at 03:35
  • @Praxeolithic: If `moduleC` is a subpackage of both, it will probably work already, since each will import its own one. – BrenBarn Nov 20 '14 at 03:40
  • To be specific, moduleA is the python portion of Pymol, moduleB is pdb, and moduleC is cmd. So I shouldn't have said both are a subpackage. – Praxeolitic Nov 20 '14 at 03:40
  • It depends on how the modules are doing their imports (e.g., with relative or absolute imports). There is no general solution to the problem, but in Python 2 relative imports should mean that it will work as long as a package doesn't try to import a global module when that same package has a submodule of the same name. Can you provide a simple toy example of a package structure showing the problem? – BrenBarn Nov 20 '14 at 03:51
  • @BrenBarn Details and progress added. – Praxeolitic Nov 20 '14 at 04:19
  • I think the problem there is not just the package structure but the fact that you are running `__init__.py` as a script; this places its directory at the front of `sys.path`. Is running `__init__.py` directly really what you're supposed to do? That is not usually a good practice. – BrenBarn Nov 20 '14 at 04:30
  • Ah, so that's why sys.path seemed to be acting strange. Launching this way is indeed what the comments in __init__.py say to do and is what a homebrew installed pymol does. It's weird. – Praxeolitic Nov 20 '14 at 04:37
  • But if you're running that file directly, how are you invoking pdb? – BrenBarn Nov 20 '14 at 04:42

1 Answers1

2

Your problem is not just with the imports, but with the fact that you are running __init__.py as a script. When you run a script, Python adds the directory containing the script to the front of sys.path, and this globally affects all subsequent imports.

There is no way to customize anything if you are directly running a file that you don't want to modify. You can't do any sneaky sys.path manipulations unless you get to run your own code first, to set up the path the way you want it. If you import the file instead of running it, you have the possibility to use your own code to tweak the paths.

I suspect that this problem is to some degree specific to PyMOL, which unfortunately does not seem to be well-designed in this regard. Looking at the source code here, I see that PyMOL's __init__.py includes a lot of custom code that does weird things like import __main__ and checking whether the version of itself that is running has various attributes. You could try using the "unsupported/experimental" method described in the comments in that file, which involves importing PyMOL instead of running it. I don't know anything about PyMOL so I don't know how that would work.

It might be worth contacting the authors of PyMOL to suggest that they fix this.

BrenBarn
  • 242,874
  • 37
  • 412
  • 384
  • Dropping code at the top of `__init__.py` would be fine. That is in fact what I'm trying at the moment. If running code before the rest of `__init__.py` is an option, what would be the appropriate sneaky sys.path manipulation? – Praxeolitic Nov 20 '14 at 04:55
  • What I now have in the OP seems to get the imports correct but like you explain, it's necessary to get a bit messy and do surgery on __init__.py. – Praxeolitic Nov 20 '14 at 05:08
  • @Praxeolitic: Yes, apparently so. Also, I really can't say how/whether that will work in combination with all the other crazy stuff that the real `__init__.py` is doing. – BrenBarn Nov 20 '14 at 05:10