2

I have an abstract base class with a number of derived classes. I'm trying to achieve the same behaviour that I would get by placing all the derived classes in the same file as the base class, i.e. if my classes are Base, DerivedA, DerivedB, DerivedC in the file myclass.py I can write in another file

import myclass
a = myclass.DerivedA()
b = myclass.DerivedB()
c = myclass.DerivedC()

but with each derived class in its own file. This has to be dynamic, i.e. such that I could e.g. delete derived_c.py and everything still works except that now I can no longer call myclass.DerivedC, or that if I add a derived_d.py, I could use it without touching the __init__.py so simply using from derived_c import DerivedC is not an option.

I've tried placing them all in a subdirectory and in that directory's __init__.py use pkgutil.walk_packages() to import all the files dynamically, but I can't get them to then be directly in the module's namespace, i.e. rather than myclass.DerivedC() I have to call myclass.derived_c.DerivedC() because I can't figure out how (or if it's possible) to use importlib to achieve the equivalent of a from xyz import * statement.

Any suggestions for how I could achieve this? Thanks!

Edit: The solutions for Dynamic module import in Python don't provide a method for automatically importing the classes in all modules into the namespace of the package.

JohnO
  • 777
  • 6
  • 10
  • Possible duplicate of [Dynamic module import in Python](https://stackoverflow.com/questions/301134/dynamic-module-import-in-python) – BugHunter Apr 30 '18 at 10:02

2 Answers2

4

I had to make something quite similar a while back, but in my case I had to dynamically create a list with all subclasses from a base class in a specific package, so in case you find it useful:

  1. Create a my_classes package containing all files for your Base class and all subclasses. You should include only one class in each file.

  2. Set __all__ appropriately in __init__.py to import all .py files except for __init__.py (from this answer):

    from os import listdir
from os.path import dirname, basename

__all__ = [basename(f)[:-3] for f in listdir(dirname(__file__)) if f[-3:] == ".py" and not f.endswith("__init__.py")]

  3. Import your classes using from my_classes import *, since our custom __all__ adds all classes inside the my_classes package to the namespace.

  4. However, this does not allow us direct access to the subclasses yet. You have to access them like this in your main script:

    from my_classes import *
from my_classes.base import Base

subclasses = Base.__subclasses__()

Now subclasses is a list containing all classes that derive from Base.

agubelu
  • 399
  • 1
  • 7
  • Thanks, unfortunately not quite what I'm looking for but may come in handy at another occasion. – JohnO Apr 30 '18 at 11:47
  • @JohnO check out the question from the answer I linked, you might find something useful there – agubelu Apr 30 '18 at 11:50
  • I found something in one of the questions linked in the one you linked, specifically https://stackoverflow.com/questions/14426574/how-to-import-members-of-modules-within-a-package/14428820#14428820. However with the caveat that both `DerivedA()` and `derived_a.DerivedA()` are now available in the package namespace. Any ideas? – JohnO Apr 30 '18 at 13:32
  • Thanks, this solution worked perfectly for what I was trying to do – Sebastian Ziegler May 13 '20 at 14:52
  • This also worked perfectly for what I was trying to do. Thank you! – zenalc Mar 09 '21 at 02:26
0

Since Python 3.6 there exists a method for initializing subclasses. This is done on definition, so before all of your code gets executed. In here you can simply import the sub-class that is initialized.

base.py

class Base:
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        __import__(cls.__module__)

sub1.py

class Sub1(Base):
    pass

sub2.py

class Sub2(Base):
    pass
Moe
  • 62
  • 10