How to implement virtual methods in Python?

Question

I know virtual methods from PHP or Java.

How can they be implemented in Python?

Or have I to define an empty method in an abstract class and override it?

Answer 1

Sure, and you don't even have to define a method in the base class. In Python methods are better than virtual - they're completely dynamic, as the typing in Python is duck typing.

class Dog:
  def say(self):
    print "hau"

class Cat:
  def say(self):
    print "meow"

pet = Dog()
pet.say() # prints "hau"
another_pet = Cat()
another_pet.say() # prints "meow"

my_pets = [pet, another_pet]
for a_pet in my_pets:
  a_pet.say()

Cat and Dog in Python don't even have to derive from a common base class to allow this behavior - you gain it for free. That said, some programmers prefer to define their class hierarchies in a more rigid way to document it better and impose some strictness of typing. This is also possible - see for example the abc standard module.

Answer 2

raise NotImplementedError(): dynamic type checking

This is the recommended exception to raise on "pure virtual methods" of "abstract" base classes that don't implement a method.

https://docs.python.org/3.5/library/exceptions.html#NotImplementedError says:

This exception is derived from RuntimeError. In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method.

As others said, this is mostly a documentation convention and is not required, but this way you get a more meaningful exception than a missing attribute error.

dynamic.py

class Base(object):
    def virtualMethod(self):
        raise NotImplementedError()
    def usesVirtualMethod(self):
        return self.virtualMethod() + 1

class Derived(Base):
    def virtualMethod(self):
        return 1

print Derived().usesVirtualMethod()
Base().usesVirtualMethod()

gives:

2
Traceback (most recent call last):
  File "./dynamic.py", line 13, in <module>
    Base().usesVirtualMethod()
  File "./dynamic.py", line 6, in usesVirtualMethod
    return self.virtualMethod() + 1
  File "./dynamic.py", line 4, in virtualMethod
    raise NotImplementedError()
NotImplementedError

typing.Protocol: static type checking (Python 3.8)

Python 3.8 added typing.Protocol which now allows us to also statically type check that a virtual method is implemented on a subclass.

protocol.py

from typing import Protocol

class CanFly(Protocol):
    def fly(self) -> str:
        pass

    def fly_fast(self) -> str:
        return 'CanFly.fly_fast'

class Bird(CanFly):
    def fly(self):
        return 'Bird.fly'

    def fly_fast(self):
        return 'Bird.fly_fast'

class FakeBird(CanFly):
    pass

assert Bird().fly() == 'Bird.fly'
assert Bird().fly_fast() == 'Bird.fly_fast'
# mypy error
assert FakeBird().fly() is None
# mypy error
assert FakeBird().fly_fast() == 'CanFly.fly_fast'

If we run this file, the asserts pass, as we didn't add any dynamic typechecking:

python protocol.py

but if we typecheck if mypy:

python -m pip install --user mypy
mypy protocol.py

we get an error as expected:

protocol.py:22: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol.py:24: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"

It is a bit unfortunate however that the error checking only picks up the error on instantiation, and not at class definition.

typing.Protocol counts methods as abstract when their body is "empty"

I'm not sure what they count as empty, but both all of the following count as empty:

• pass
• ... ellipsis object
• raise NotImplementedError()

So the best possibility is likely:

protocol_empty.py

from typing import Protocol

class CanFly(Protocol):
    def fly(self) -> None:
        raise NotImplementedError()

class Bird(CanFly):
    def fly(self):
        return None

class FakeBird(CanFly):
    pass

Bird().fly()
FakeBird().fly()

which fails as desired:

protocol_empty.py:15: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol_empty.py:15: note: The following method was marked implicitly abstract because it has an empty function body: "fly". If it is not meant to be abstract, explicitly return None.

but if e.g. we replace the:

raise NotImplementedError()

with some random "non-empty" statement such as:

x = 1

then mypy does not count them as virtual and gives no errors.

@abc.abstractmethod: dynamic + static + documentation in one go

Previously mentioned at: https://stackoverflow.com/a/19316077/895245 but the metaclass syntax changed in Python 3 to:

class C(metaclass=abc.ABCMeta):

instead of the Python 2:

class C:
    __metaclass__=abc.ABCMeta

so now to use @abc.abstractmethod which was previously mentioned at https://stackoverflow.com/a/19316077/895245 you need:

abc_cheat.py

import abc

class C(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def m(self, i):
        pass

try:
    c = C()
except TypeError:
    pass
else:
    assert False

Vs raise NotImplementedError and Protocol:

• disadvantage: more verbose
• advantage: does all of dynamic checks, static checks and shows up on documentation (see below)

https://peps.python.org/pep-0544 does mention both approaches in passing

E.g.:

abc_bad.py

#!/usr/bin/env python

import abc

class CanFly(metaclass=abc.ABCMeta):
    '''
    doc
    '''

    @abc.abstractmethod
    def fly(self) -> str:
        '''
        doc
        '''
        pass

class Bird(CanFly):
    '''
    doc
    '''

    def fly(self):
        '''
        doc
        '''
        return 'Bird.fly'

class Bat(CanFly):
    '''
    doc
    '''
    pass

def send_mail(flyer: CanFly) -> str:
    '''
    doc
    '''
    return flyer.fly()

assert send_mail(Bird()) == 'Bird.fly'
assert send_mail(Bat()) == 'Bat.fly'

then:

mypy abc_bad.py

fails as desired with:

main.py:40: error: Cannot instantiate abstract class "Bat" with abstract attribute "fly"

Sphinx: make it show on the documentation

See: How to annotate a member as abstract in Sphinx documentation?

Of the methods mentioned above, only one shows up on the sphinx documentation output: @abc.abstractmethod.

Outro

Bibiography:

• https://peps.python.org/pep-0544 the typing.Protocol PEP
• Is it possible to make abstract classes?
• What to use in replacement of an interface/protocol in python

Tested on Python 3.10.7, mypy 0.982, Ubuntu 21.10.

Answer 3

Python methods are always virtual.

Answer 4

Actually, in version 2.6 python provides something called abstract base classes and you can explicitly set virtual methods like this:

from abc import ABCMeta
from abc import abstractmethod
...
class C:
    __metaclass__ = ABCMeta
    @abstractmethod
    def my_abstract_method(self, ...):

It works very well, provided the class does not inherit from classes that already use metaclasses.

source: http://docs.python.org/2/library/abc.html

Answer 5

Python methods are always virtual

like Ignacio said yet Somehow class inheritance may be a better approach to implement what you want.

class Animal:
    def __init__(self,name,legs):
        self.name = name
        self.legs = legs

    def getLegs(self):
        return "{0} has {1} legs".format(self.name, self.legs)

    def says(self):
        return "I am an unknown animal"

class Dog(Animal): # <Dog inherits from Animal here (all methods as well)

    def says(self): # <Called instead of Animal says method
        return "I am a dog named {0}".format(self.name)

    def somethingOnlyADogCanDo(self):
        return "be loyal"

formless = Animal("Animal", 0)
rover = Dog("Rover", 4) #<calls initialization method from animal

print(formless.says()) # <calls animal say method

print(rover.says()) #<calls Dog says method
print(rover.getLegs()) #<calls getLegs method from animal class

Results should be:

I am an unknown animal
I am a dog named Rover
Rover has 4 legs

Answer 6

Something like a virtual method in C++ (calling method implementation of a derived class through a reference or pointer to the base class) doesn't make sense in Python, as Python doesn't have typing. (I don't know how virtual methods work in Java and PHP though.)

But if by "virtual" you mean calling the bottom-most implementation in the inheritance hierarchy, then that's what you always get in Python, as several answers point out.

Well, almost always...

As dplamp pointed out, not all methods in Python behave like that. Dunder method don't. And I think that's a not so well known feature.

Consider this artificial example

class A:
    def prop_a(self):
        return 1
    def prop_b(self):
        return 10 * self.prop_a()

class B(A):
    def prop_a(self):
        return 2

Now

>>> B().prop_b()
20
>>> A().prob_b()
10

However, consider this one

class A:
    def __prop_a(self):
        return 1
    def prop_b(self):
        return 10 * self.__prop_a()

class B(A):
    def __prop_a(self):
        return 2

Now

>>> B().prop_b()
10
>>> A().prob_b()
10

The only thing we've changes was making prop_a() a dunder method.

A problem with the first behavior can be that you can't change the behavior of prop_a() in the derived class without impacting the behavior of prop_b(). This very nice talk by Raymond Hettinger gives an example for a use case where this is inconvenient.

Answer 7

Python 3.6 introduced __init_subclass__ and this let you simply do this:

class A:

    def method(self):
        '''method needs to be overwritten'''
        return NotImplemented

    def __init_subclass__(cls):
        if cls.method is A.method:
            raise NotImplementedError(
                'Subclass has not overwritten method {method}!')

The benefit of this solution is that you avoid the abc metaclass and give the user a direct imperative how to do it right. In addition to another answer here that raises NotImplementedError when calling the method. This solution is checked on runtime and not only IF the user calls the method.

Answer 8

Virtual is not same as implementing abstract in inheriting child class.

Virtual supposed to be used with Interface contract, where you can give the code a bit extra flexibility and still gain the benefits of low-code duplication since the parent class can implement the basic function implementation that applies to 9/10 of your classes and through virtual you can override the 1/10 without needing to duplicate your functions in child-classes x,y or n because of the abstract definition would require you to do*.

I doubt there is any robust solution to support virtual in Python unluckily, as even the interfaces are not yet as part of the built-in modules (abc module, even often used to imitate interfaces, cannot really be considered as proper interface support).

*Software design quality-wise speaking it is one technique that can be used to move code towards low coupling, high cohesion quality goals, that lead to anti-spaghetti and eventually to code generation and fully generic modular coding philosophy, that is written as an generative engine, rather than custom implementation. There are many OOP programmers who claim OOP cannot be generated from the scratch due its "nature", the OOP is not supposed to be anything else than generated actually, especially if you do all of it before runtime (generate the whole runtime code in between), the better.

In case of instantiating new objects, you can enable high perf just like done in graphic rendering libraries. Generating an object blitter, to handle sudden needs for objects + on top library that handles management for such/orchestrates such real-time performance critical needs. Also, depending on your use-case, you can store them in high-perf data-structures such as Stacks, or List/Dict of Stacks incase you need to group them.