Python 3 - Method docstring inheritance without breaking decorators or violating DRY

Question

This question seems to come up regularly both on StackOverflow and elsewhere, yet I wasn't able to find a completely satisfactory solution anywhere.

There seem to be two types of common solutions. The first one (from e.g. http://article.gmane.org/gmane.comp.python.general/630549) uses a function decorator:

class SuperClass:
    def my_method(self):
        '''Has a docstring'''
        pass

class MyClass(SuperClass):
    @copy_docstring_from(SuperClass)
    def my_method(self):
        pass

assert SuperClass.my_method.__doc__ == MyClass.my_method._doc__

This is probably the most straightforward approach, but it requires repeating the parent class name at least once, and also becomes a lot more complicated if the docstring can not be found in the direct ancestor.

The second approach uses a metaclass or class decorator (cf. Inheriting methods' docstrings in Python, Inherit a parent class docstring as __doc__ attribute, http://mail.python.org/pipermail/python-list/2011-June/606043.html) and looks like this:

class MyClass1(SuperClass, metaclass=MagicHappeningHere):
    def method(self):
        pass

# or 

@frobnicate_docstrings
class MyClass2(SuperClass):
    def method(self):
        pass

assert SuperClass.my_method.__doc__ == MyClass1.my_method._doc__
assert SuperClass.my_method.__doc__ == MyClass2.my_method._doc__

However, with this approach the docstring is only set after class creation and thus not accessible to decorators, so the following won't work:

def log_docstring(fn):
    print('docstring for %s is %s' % (fn.__name__, fn.__doc__)
    return fn

class MyClass(SuperClass, metaclass=MagicHappeningHere):
# or
#@frobnicate_docstrings
#class MyClass2(SuperClass): 
    @log_docstring
    def method(self):
        pass

A third interesting idea has been discussed in Inherit docstrings in Python class inheritance. Here, the function decorator actually wraps the method and turns it into a method descriptor rather than merely updating its docstring. However, this seems like using sledgehammer to crack a nut because it turns the method into a method descriptor (which may have performance implications as well, though I did not check), and also does not make the docstring available to any other decorators (and in the above example will actually make them crash because the method descriptor doesn't have __name__ attribute).

Is there a solution that avoids all the above drawbacks, i.e. does not require me to repeat myself and assigns the docstring immediately using a decorator?

I'm interested in a solution for Python 3.

Answer 1

Starting in Python 3.5, inspect.getdoc searches the inheritance tree for a docstring. So if you leave the docstring for the child empty, it will retrieve it from the parent. That avoids the need for code repetition, and automatic code generators like sphinx will do the right thing.

$ cat mwe.py
import inspect

class A:
    def foo(self):
        """Fool!"""
        return 42

class B(A):
    def foo(self):
        return super().foo()

print(A.foo.__doc__, B.foo.__doc__, A().foo.__doc__, B().foo.__doc__,
      inspect.getdoc(A.foo), inspect.getdoc(B.foo),
      inspect.getdoc(A().foo), inspect.getdoc(B().foo))
$ python mwe.py
Fool! None Fool! None Fool! Fool! Fool! Fool!

Answer 2

Use a class decorator instead:

@inherit_docstrings
class MyClass(SuperClass):
    def method(self):
        pass

where inherit_docstrings() is defined as:

from inspect import getmembers, isfunction

def inherit_docstrings(cls):
    for name, func in getmembers(cls, isfunction):
        if func.__doc__: continue
        for parent in cls.__mro__[1:]:
            if hasattr(parent, name):
                func.__doc__ = getattr(parent, name).__doc__
    return cls

Demo:

>>> class SuperClass:
...     def method(self):
...         '''Has a docstring'''
...         pass
... 
>>> @inherit_docstrings
... class MyClass(SuperClass):
...     def method(self):
...         pass
... 
>>> MyClass.method.__doc__
'Has a docstring'

This sets the docstring after defining the whole class, without having to create an instance first.

If you need the docstring available to method decorators, you are, unfortunately, wholly stuck with your decorator that duplicates the parent class.

The reason for this is that you cannot introspect what the superclass is going to be while defining the class body. The local namespace during class definition does not have access to the arguments passed to the class factory.

You could use a metaclass to add the base classes to the local namespace, then use a decorator to pull those out again, but in my opinion that gets ugly, fast:

import sys

class InheritDocstringMeta(type):
    _key = '__InheritDocstringMeta_bases'

    def __prepare__(name, bases, **kw):
        return {InheritDocstringMeta._key: bases}

    def __call__(self, name, bases, namespace, **kw):
        namespace.pop(self._key, None)

def inherit_docstring(func):
    bases = sys._getframe(1).f_locals.get(InheritDocstringMeta._key, ())
    for base in bases:
        for parent in base.mro():
            if hasattr(parent, func.__name__):
                func.__doc__ = getattr(parent, func.__name__).__doc__
    return func

Demo usage:

>>> class MyClass(SuperClass, metaclass=InheritDocstringMeta):
...     @inherit_docstring
...     def method(self):
...         pass
... 
>>> MyClass.method.__doc__
'Has a docstring'

Answer 3

I think the metaclass' __prepare__ method can be used for this by injecting a decorator that knows about the class hierarchy:

def log_docstring(fn):
    print('docstring for %r is %r' % (fn, fn.__doc__))
    return fn

class InheritableDocstrings(type):
    def __prepare__(name, bases):
        classdict = dict()

        # Construct temporary dummy class to figure out MRO
        mro = type('K', bases, {}).__mro__[1:]
        assert mro[-1] == object
        mro = mro[:-1]

        def inherit_docstring(fn):
            if fn.__doc__ is not None:
                raise RuntimeError('Function already has docstring')

            # Search for docstring in superclass
            for cls in mro:
                super_fn = getattr(cls, fn.__name__, None)
                if super_fn is None:
                    continue
                fn.__doc__ = super_fn.__doc__
                break
            else:
                raise RuntimeError("Can't inherit docstring for %s: method does not "
                                   "exist in superclass" % fn.__name__)

            return fn

        classdict['inherit_docstring'] = inherit_docstring
        return classdict

class Animal():
    def move_to(self, dest):
        '''Move to *dest*'''
        pass

class Bird(Animal, metaclass=InheritableDocstrings):
    @log_docstring
    @inherit_docstring
    def move_to(self, dest):
        self._fly_to(dest)

assert Animal.move_to.__doc__ == Bird.move_to.__doc__

Prints:

docstring for <function Bird.move_to at 0x7f6286b9a200> is 'Move to *dest*'

Of course, this approach has some other issues: - Some analysis tools (e.g. pyflakes) will complain about the use of the (apparently) undefined inherit_docstring name - It doesn't work if the parent class already has a different metaclass (e.g. ABCMeta).