Dynamically imported module doesn't think it has class

Question

Setup: Python 3.3

I'm making an application that looks through a folder called 'sources' for .py files, and look in them to find classes that extend a class called 'SourceBase' that I defined. If they extend SourceBase, I want to make a new instance of the class to work on.

I've done some fair amount of research through the following posts, which I understand for the most part:

My folder setup is like this, which I beleive is relevant:

EPDownloader [package]
\
 epdownloader.py [main]
 SourceBase.py [contains SourceBase class]
 imageutils.py [this class will find and dynamically load the classes in the sources package]
 sources [package]
 \
  source1.py [has class that extends SourceBase]
  source2.py
  ...other plugins here...

My issue is that I'm using the following code (from the other stack overflow questions I listed above) and it's searching through my module for classes, but it doesn't find my classes. It just skips them. I'm not sure what's wrong. Here is my code that is performing the search (its based off the first link I posted):

<!--language: python-->
def getSources(self):
    pluginbase=SourceBase.SourceBase
    searchpath='sources'
    #We want to iterate over all modules in  the sources/ directory, allowing the user to make their own.
    for root, dirs, files in os.walk('./'+searchpath):
        print('files: ',files)
        candidates = [fname for fname in files if fname.endswith('.py') 
                      and not fname.startswith('__')]
        classList=[]
        if candidates:
            for c in candidates:
                modname = os.path.splitext(c)[0]
                print('importing: ',modname)
                module=__import__(searchpath+'.'+modname)   #<-- You can get the module this way
                print('Parsing module '+modname)
                for cls in dir(module):          #<-- Loop over all objects in the module's namespace
                    print('Inspecting item from module: '+str(cls))
                    cls=getattr(module,cls) #this seems to still be a module when it hits source1
                    print('Get attribute: '+str(cls))
                    if (inspect.isclass(cls)):                # Make sure it is a class
                        print('...is a class') 
                        if inspect.getmodule(cls)==module:  # Make sure it was defined in module, not just imported
                            print('...is in a module')
                            if issubclass(cls,pluginbase):          # Make sure it is a subclass of base
                                print('...subclasses '+pluginbase.__name__)
                                classList.append(cls)
        print(classList)

Here is the relevant output it gives me (I trimmed a lot of other stuff this code outputs):

Inspecting item from module: source1
Get attribute: <module 'sources.source1' from    '/Users/Mgamerz/Documents/workspace/code/EPDownloader/sources/source1.py'>
[] <--- signifies it failed to find the source class

I'm pretty sure my subclassing works, here's a snippet of the class:

from EPDownloader import SourceBase
class source1(SourceBase.SourceBase):
    def __init__(self):
        pass

I'm stumped by this problem. I've spent the last few hours on it and I don't know what to do. I have a feeling its a simple fix I'm not seeing. Can someone help me find the bug here?

[Note: I looked through the StackOverflow formatting help, and don't see any way to format a 'highlight', the ones where it puts a grey background on text, but inline. It would help highlight parts of this question I'm trying to convey.]

score 2 · Accepted Answer · answered Jan 12 '14 at 22:06

Look at the documentation: http://docs.python.org/3.1/library/functions.html#import

When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name. However, when a non-empty fromlist argument is given, the module named by name is returned.

Simply replace

module=__import__(searchpath+'.'+modname)

with

module=__import__(searchpath+'.'+modname, None, None, "*")

It's the same as "from sources.source1 import *" which tells __import__ to fetch everything inside the given module.

I had to debug a bit more, but your soultion worked. I've been bashing my head on this for hours. Thanks. — Mgamerz, Jan 12 '14 at 23:09

score 1 · Answer 2 · answered Jan 12 '14 at 22:06

There is something wrong with your __import__: instead of importing a module, you're importing the whole package (the whole 'sources' directory as a package).

I could fix your code doing this:

for c in candidates:
        modname = os.path.splitext(c)[0]
        print('importing: ',modname)
        # NEW CODE
        sys.path.insert(0, searchpath)
        module=__import__(modname)   #<-- You can get the module this way
        # END OF NEW CODE
        print('Parsing module '+modname)
        ...

Dynamically imported module doesn't think it has class

2 Answers2