When inheriting SQLAlchemy class from abstract class exception thrown: metaclass conflict: the metaclass of a derived class must be

Question

The following code is a very simple implementation of a SqlAlchemy ORM with one simple table. The Mytable class tries to inherit from BaseAbstract.

The code throws the following exception:

Message: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

from abc import ABC
from sqlalchemy import Column, Integer, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

class BaseAbstract(ABC):
    """description of class"""

SQLALCHEMY_DATABASE_URI =\
   'mssql+pyodbc://(local)/TestDB?driver=SQL+Server+Native+Client+11.0'
SQLALCHEMY_TRACK_MODIFICATIONS = False

engine = create_engine(SQLALCHEMY_DATABASE_URI, echo=True)
Session = sessionmaker(bind=engine)
session = Session()

Base = declarative_base()
metadata = Base.metadata

class Mytable(Base, BaseAbstract):
    __tablename__ = 'myTable'

    id = Column(Integer, primary_key=True)
    firstNum = Column(Integer, nullable=False)
    secondNum = Column(Integer, nullable=False)

If you change the class declaration line to

class Mytable(Base):

the code will work fine. Also if you change class BaseAbstract(ABC): to class BaseAbstract(object): the code will again work fine. How do I inherit from an abstract class in SQLAlchemy?

Answer 1

Mixing metaclasses is not easy and you should avoid it. SQLAlchemy offers a way to handle abstract base classes or augmenting the base, and on the other hand what you're trying to do looks a lot like a mixin.

You can instruct SQLAlchemy to skip creating a table and a mapper for a class using __abstract__:

Base = declarative_base()

class BaseAbstract(Base):
    """description of class"""
    __abstract__ = True

class Mytable(BaseAbstract):
    ...

You could also augment the Base class:

class BaseAbstract:
    """description of class"""

Base = declarative_base(cls=BaseAbstract)

class Mytable(Base):
    ...

But in my opinion the easiest solution is to forego using an "abstract base" altogether and think of it as a mixin, as you had done already in a way:

class CommonMixin:
    """description of class"""

Base = declarative_base()

class Mytable(CommonMixin, Base):
    ...

But if you insist on using an actual abc.ABC abstract base class, register your model classes as virtual subclasses:

class BaseAbstract(ABC):
    """description of class"""

Base = declarative_base()

@BaseAbstract.register
class Mytable(Base):
    ...

The downside is that @abc.abstractmethod decorated methods are not checked upon instantiating virtual subclasses.

If the above do not fulfill your needs and you want to use ABC for checking that required methods are implemented, you could try and do as the exception instructed and create a new metaclass that is the combination of DeclarativeMeta and ABCMeta:

In [6]: class DeclarativeABCMeta(DeclarativeMeta, abc.ABCMeta):
   ...:     pass
   ...: 

In [7]: Base = declarative_base(metaclass=DeclarativeABCMeta)

In [8]: class BaseAbstract(abc.ABC):
   ...:     @abc.abstractmethod
   ...:     def foo(self):
   ...:         pass
   ...:     

In [13]: class MyTable(Base, BaseAbstract):
    ...:     __tablename__ = 'mytable'
    ...:     id = Column(Integer, primary_key=True)
    ...:     

In [14]: MyTable()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-1686a36a17c6> in <module>()
----> 1 MyTable()

TypeError: "Can't instantiate abstract class MyTable with abstract methods foo"

In [18]: class MyOtherTable(Base, BaseAbstract):
    ...:     __tablename__ = 'myothertable'
    ...:     id = Column(Integer, primary_key=True)
    ...:     def foo(self):
    ...:         return 'bar'
    ...:     

In [19]: MyOtherTable()
Out[19]: <__main__.MyOtherTable at 0x7f01b4b592b0>

I cannot vouch for this, though. It might contain more than a few surprises.

Answer 2

I came across the same problem. In addition to the very good answer of Ilja and just because in a comment on their answer you are asking

The reason I am trying to inherit from an abstract class inheriting from ABC is to use isinstance in a function that takes as an argument an object and if that returns true then do additional processing by calling methods defined in the abstract class. If I do not inherit from ABC, Python will not enforce my need to make sure the inherited class implements all the attributes of the abstract class, and if I have missed some in an implementation, then I will get a run-time exception. I would rather get a build exception. That is what ABC gives me. Is this not a good reason to use ABC?

I would like to share the workaround I created before I came here. Besides ABC, Python provides another way to check for the presence of a certain method during class instantiation, i.e. in the __init_subclass__ magic method. I did the following:

class BaseAbstract:
    ABSTRACT_METHODS = [
        "some_method", "another_method"
    ]

    def __init_subclass__(cls):
        for method_name in BaseAbstract.ABSTRACT_METHODS:
            method = getattr(cls, method_name, None)
            if not method or not callable(method):
                raise TypeError(
                    f"Can't instantiate abstract class {cls.__name__} with "
                    f"abstract methods {method_name}."
                )

Using inspect, you could also check for constraints on the signatures of the "abstract" methods.

It is a matter of argument whether this is the better way to go than using one of Ilja's ideas. I just want to share my approach, not claiming that it was superior.

I see two neat characteristics of my workaround:

1. I have only one SQLalchemy Base. So I don't have to worry about mutiple Bases, for instance when using alembic's feature to autogenerate migrations or when calling Base.metadata.create_all()

2. I do not actually have to understand what the problem was. If you have never dealt with metadata conflicts, with my approach is easier to understand why it fixes the problem.

But still, it could be considered some sort of hacky. It also has the characteristic of checking for method presence during class initialization, not during instantiation. This is a limitation, in case you have cascading inheritances, where all the methods are present on the last child, but not on all intermediate classes in the inheritance chain.

Answer 3

You should be able to solve the issue by introducing an intermediate base class for your Mytable. For example:

Base = declarative_base()
metadata = Base.metadata

class IntermediateBase(type(Base), BaseAbstract):
    pass

class Mytable(IntermediateBase):
    __tablename__ = 'myTable'

    id = Column(Integer, primary_key=True)
    firstNum = Column(Integer, nullable=False)
    secondNum = Column(Integer, nullable=False)

Answer 4

Following one of Ilja Everilä's examples above was really useful for me. Mixing metaclasses can be tricky but I haven't noticed any issue yet. I actually subclass the declarative_base into my parent class where common functions, attributes, abstractmethods can be defined

from abc import abstractmethod, ABCMeta
from typing import Dict, Any

from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base, DeclarativeMeta


class DeclarativeABCMeta(DeclarativeMeta, ABCMeta):
    """
    AbstractBaseClass Metaclass

    This allows us to enforce the @abstractmethods defined on MLRegistryBase
    on the underlying sub-classes (Database Tables)
    """
    pass


Base = declarative_base(metaclass=DeclarativeABCMeta)


class TableAbstractParentClass(Base):
    """
    Helper Class for Database Model Inheritance
    """

    # Inform SQLAlchemy This in an AbstractBaseClass Model
    __abstract__ = True

    @classmethod
    @abstractmethod
    def __tablename__(cls) -> str:
        """
        Every Database Table Needs a Name
        """
        pass

    @abstractmethod
    def to_dict(self) -> Dict[str, Any]:
        """
        Example Required Method
        """
        pass


class ExampleTable(TableAbstractParentClass):
    """
    Example Table
    """
    __tablename__ = "example_table"

    id = Column(Integer, unique=True, autoincrement=True,
                primary_key=True, index=True)
    name = Column(String(256), nullable=False)

    def to_dict(self) -> Dict[str, Any]:
        return {
            "id": self.id,
            "name": self.name
        }