How to implement an interface in a way that is compatible with static type checks?

Question

I have two base classes, Foo and Bar, and a Worker class which expects objects that behave like Foo. Then I add another class that implements all relevant attributes and methods from Foo but I didn't manage to communicate this successfully to static type checking via mypy. Here's a small example:

class MyMeta(type):
    pass

class Bar(metaclass=MyMeta):
    def bar(self):
        pass

class Foo:
    def __init__(self, x: int):
        self.x = x

    def foo(self):
        pass

class Worker:
    def __init__(self, obj: Foo):
        self.x = obj.x

Here Worker actually accepts any Foo-ish object, i.e. objects that have an attribute x and a method foo. So if obj walks like a Foo and if it quacks like a Foo then Worker will be happy. Now the whole project uses type hints and so for the moment I indicate obj: Foo. So far so good.

Now there's another class FooBar, which subclasses Bar and behaves like Foo but it can't subclass Foo because it exposes its attributes via properties (and so the __init__ parameters don't make sense):

class FooBar(Bar):
    """Objects of this type are bar and they are foo-ish."""

    @property
    def x(self) -> int:
        return 0

    def foo(self):
        pass

At this point, doing Worker(FooBar()) obviously results in a type checker error:

error: Argument 1 to "Worker" has incompatible type "FooBar"; expected "Foo"

Using an abstract base class

In order to communicate the interface of Foo-ish to the type checker I thought about creating an abstract base class for Foo-ish types:

import abc

class Fooish(abc.ABC):
    x : int

    @abc.abstractmethod
    def foo(self) -> int:
        raise NotImplementedError

However I can't make FooBar inherit from Fooish because Bar has its own metaclass and so this would cause a metaclass conflict. So I thought about using Fooish.register on both Foo and FooBar but mypy doesn't agree:

@Fooish.register
class Foo:
    ...

@Fooish.register
class FooBar(Bar):
    ...

class Worker:
    def __init__(self, obj: Fooish):
        self.x = obj.x

Which produces the following errors:

error: Argument 1 to "Worker" has incompatible type "Foo"; expected "Fooish"
error: Argument 1 to "Worker" has incompatible type "FooBar"; expected "Fooish"

Using a "normal" class as an interface

The next option I considered is creating an interface without inheriting from abc.ABC in form of a "normal" class and then have both Foo and FooBar inherit from it:

class Fooish:
    x : int

    def foo(self) -> int:
        raise NotImplementedError

class Foo(Fooish):
    ...

class FooBar(Bar, Fooish):
    ...

class Worker:
    def __init__(self, obj: Fooish):
        self.x = obj.x

Now mypy doesn't complain about the argument type to Worker.__init__ but it complains about signature incompatibility of FooBar.x (which is a property) with Fooish.x:

error: Signature of "x" incompatible with supertype "Fooish"

Also the Fooish (abstract) base class is now instantiable and a valid argument to Worker(...) though it doesn't make sense since it doesn't provide an attribute x.

The question ...

Now I'm stuck at the question on how to communicate this interface to the type checker without using inheritance (due to metaclass conflict; even if it was possible, mypy would still complain about signature incompatibility of x). Is there a way to do it?

Answer 1

Support for structural subtyping was added by PEP 544 -- Protocols: Structural subtyping (static duck typing) starting with Python 3.8. For versions prior to 3.8 the corresponding implementation is made available by the typing-extensions package on PyPI.

Relevant for the discussed scenario is typing.Protocol as explained by the PEP in more detail. This allows to define implicit subtypes which saves us from the metaclass conflict issue since inheritance is not required. So the code looks like this:

from typing import Protocol             # Python 3.8+
from typing_extensions import Protocol  # Python 3.5 - 3.7


class Fooish(Protocol):
    x : int

    def foo(self) -> int:
        raise NotImplementedError


# No inheritance required, implementing the defined protocol implicitly subtypes 'Fooish'.
class Foo:
    def __init__(self, x: int):
        self.x = x

    def foo(self):
        pass


class MyMeta(type):
    pass


class Bar(metaclass=MyMeta):
    def bar(self):
        pass


# Here, we again create an implicit subtype of 'Fooish'.
class FooBar(Bar):
    """Objects of this type are bar and they are foo-ish."""

    @property
    def x(self) -> int:
        return 0

    @x.setter
    def x(self, val):
        pass

    def foo(self):
        pass


class Worker:
    def __init__(self, obj: Fooish):
        self.x = obj.x

Answer 2

1. To get rid of error: Signature of "x" incompatible with supertype "Fooish" you can annotate x: typing.Any.
2. To make Fooish really abstract some tricks are needed to resolve metaclass conflict. I took a recipe from this answer:

class MyABCMeta(MyMeta, abc.ABCMeta):
    pass

After that it is possible to create Fooish:

class Fooish(metaclass=MyABCMeta):

The whole code that successfully executes at runtime and shows no errors from mypy:

import abc
import typing

class MyMeta(type):
    pass

class MyABCMeta(abc.ABCMeta, MyMeta):
    pass

class Fooish(metaclass=MyABCMeta):
    x : typing.Any

    @abc.abstractmethod
    def foo(self) -> int:
        raise NotImplementedError

class Bar(metaclass=MyMeta):
    def bar(self):
        pass

class Foo(Fooish):
    def __init__(self, x: int):
        self.x = x

    def foo(self):
        pass

class Worker:
    def __init__(self, obj: Fooish):
        self.x = obj.x


class FooBar(Bar, Fooish):
    """Objects of this type are bar and they are foo-ish."""

    @property
    def x(self) -> int:
        return 0

    def foo(self):
        pass

print(Worker(FooBar()))

Now it is time to think do you really want to make Fooish abstract because doing class Fooish(metaclass=MyABCMeta): can have side effects if MyMeta does lot of tricks. For example if MyMeta defines __new__ you can probably define __new__ in Fooish which doesn't call MyMeta.__new__ but calls abc.ABCMeta.__new__. But things can become complicated... So, maybe it will be easier to have non-abstract Fooish.

Answer 3

If I understand, you might be able to add a Union, which basically, allows Foo or Bar or Fooish:

from typing import Union

class Worker:
    def __init__(self, obj: Union[Bar, Fooish]):
        self.x = obj.x

# no type error
Worker(FooBar())

With the following:

class MyMeta(type):
    pass

class Fooish:
    x: int
    def foo(self) -> int:
        raise NotImplementedError


class Bar(metaclass=MyMeta):
    def bar(self):
        pass


class Foo:
    def __init__(self, x: int):
        self.x = x

    def foo(self):
        pass


class Worker:
    def __init__(self, obj: Union[Bar, Fooish]):
        self.x = obj.x


class FooBar(Bar, Fooish):
    """Objects of this type are bar and they are foo-ish."""

    @property
    def x(self) -> int:
        return 0

    def foo(self):
        pass

see:

• https://github.com/python/typing/issues/213