Python abc and factory function

Question

I'm trying to implement a factory function that creates subclasses of a given base class (comparing to a string of its name for now). I have this

class Base(object):
    ...

And in another file I have

class Sub(Base):
    ...

I'm keeping those classes separated into files because I'm going to have many sub classes and I don't want to define them all in one large file (I would settle for just forward-declaring if possible somehow).

Now in the "Base" file, I want to implement a function to create an instance out of a given name, but Base.__subclasses__() is None, so I can't do something like this:

def factory(name):
    for Subclass in Base.__subclasses__()
        if name is ...
            return Subclass()

My question is what is the best approach to implement this kind of pattern. In the future I'd like to change this to maybe use a hashtable, but I can't get this simple "string based" example to work.

there are no declerations in python, except for `globals` and `nonlocals` — Elazar, May 31 '13 at 11:56
When are you calling `factory`? If it is *after* `Sub` is defined, then `Base.__subclasses__()` will include `Sub`. — unutbu, May 31 '13 at 11:57
The response to this question seemks relevent:http://stackoverflow.com/questions/3507125/how-can-i-discover-classes-in-a-specific-package-in-python — Tom Dalton, May 31 '13 at 11:59
@unutbu - What do you mean by "before" or "after"? These are all different files. — asafge, May 31 '13 at 12:03
You should import all modules with subclasses to "register" them. maybe You will need some kind of auto-discover function — oleg, May 31 '13 at 12:04
@oleg - What I just did what import all those SubClasses in `__init__.py` and added them to `__all__ = [...]`. Looks like `Base.__subclasses__()` is getting filled up, but is this the best approach? It sure feels like "forward declaring". — asafge, May 31 '13 at 12:17
@asafge: "before" and "after" are with respect to the order in which the statements get called. You had shown us the statements, but not the flow of control. Can you give use a runnable example? Then it might be clearer if there is a better approach. — unutbu, May 31 '13 at 12:34
@asafge Keep in mind that in mind all statements are *executable* statements. There is no such thing as "declaration". If you don't execute something then the interpreter doesn't know that it exists. In your case the subclasses *do not exist*(literally) until you import the module where you are defining them. — Bakuriu, May 31 '13 at 14:09

score 2 · Accepted Answer · edited May 23 '17 at 12:17

You should realise that your class starts to exist only when its definition is read by the interpreter. So the module with that class has to be imported first.

If all your subclasses are located in one file, just import it before doing other things, so that the definitions are read. If subclassess are in multiple files, you'll need some kind of dynamic import.

And you don't need __subclassess__ for what you described, having a name is just enough. Of course, if your ultimate goal is actually instantiating all the subclasses, you'll need it, but, again, a subclass will appear in __subclassess__ only when its definition is read, that is, its module is imported.

Thanks for all the comments. As I've learned the main thing here is to notice the flow, and make the interpreter "aware" of my sub-classes before trying to factory() them. — asafge, May 31 '13 at 16:29

score 0 · Answer 2 · answered May 31 '13 at 12:40

if what you want is the "best pattern", then consider using the entry points feature of setuptools and pkg_resources. This will probably require you use a proper setup.py to make an egg distributable package. In a file named setup.py add:

from setuptools import setup, find_packages
setup(
    name="HelloWorld",
    version="0.1",
    packages=find_packages(),
    entry_points={'my_subclasses': [
        "foo = package.module1:Foo",
        "bar = package.module2:Bar",
    ]}
)

while you're hacking, install your package with

# python setup.py develop

or, if you use virtualenv:

$ pip install -e .

Then, your factory function gets pretty simple, in fact it just calls pkg_resources to do all the work.

from pkg_resources import iter_entry_points

def factory(name):
    entry_point = iter_entry_points('my_subclasses', name)
    cls = entry_point.load()
    # create an instance?
    return cls()

There are a few advantages with this approach; the biggest is that other users of your library can provide their own implementations by installing their own eggs with entry points in the same group ('my_subclasses' in the example)

Another advantage is that the unused modules don't need to be imported, only the class which was returned by the factory needed to be imported.

If you'd rather not have to maintain the list of entry points manually, in addition to actually creating them; the setup.py is a normal python script, and can execute code to find all of the classes by importing them all or walking the filesystem.

Python abc and factory function

2 Answers2