Python Sphinx autodoc and decorated members

Question

I am attempting to use Sphinx to document my Python class. I do so using autodoc:

.. autoclass:: Bus
   :members:

While it correctly fetches the docstrings for my methods, those that are decorated:

    @checkStale
    def open(self):
        """
        Some docs.
        """
        # Code

with @checkStale being

def checkStale(f):
    @wraps(f)
    def newf(self, *args, **kwargs):
        if self._stale:
            raise Exception
        return f(self, *args, **kwargs)
    return newf

have an incorrect prototype, such as open(*args, **kwargs).

How can I fix this? I was under the impression that using @wraps would fix up this kind of thing.

Answer 1

I had the same problem with the celery @task decorator.

You can also fix this in your case by adding the correct function signature to your rst file, like this:

.. autoclass:: Bus
    :members:

    .. automethod:: open(self)
    .. automethod:: some_other_method(self, param1, param2)

It will still document the non-decorator members automatically.

This is mentioned in the sphinx documentation at http://www.sphinx-doc.org/en/master/ext/autodoc.html#directive-automodule -- search for "This is useful if the signature from the method is hidden by a decorator."

In my case, I had to use autofunction to specify the signature of my celery tasks in the tasks.py module of a django app:

.. automodule:: django_app.tasks
    :members:
    :undoc-members:
    :show-inheritance:

    .. autofunction:: funct1(user_id)
    .. autofunction:: func2(iterations)

Answer 2

To expand on my comment:

Have you tried using the decorator package and putting @decorator on checkStale? I had a similar issue using epydoc with a decorated function.

As you asked in your comment, the decorator package is not part of the standard library.

You can fall back using code something like the following (untested):

try:
    from decorator import decorator
except ImportError:
    # No decorator package available. Create a no-op "decorator".
    def decorator(f):
        return f

Answer 3

I just found an easy solution which works for me, but don't ask me why. If you know why add it in the comments.

from functools import wraps 

def a_decorator(f):
    """A decorator 

    Args:
        f (function): the function to wrap
    """
    @wraps(f) # use this annotation on the wrapper works like a charm
    def wrapper(*args, **kwargs):
        some code
        return ret

return wrapper

The doc of the decorated function and of the decorator are both kept

Answer 4

Added in version 1.1 you can now override the method signature by providing a custom value in the first line of your docstring.

http://sphinx-doc.org/ext/autodoc.html#confval-autodoc_docstring_signature

@checkStale
def open(self):
    """
    open()
    Some docs.
    """
    # Code

Answer 5

Add '.__ doc __':

def checkStale(f):
    @wraps(f)
    def newf(self, *args, **kwargs):
       if self._stale:
          raise Exception
       return f(self, *args, **kwargs)
    newf.__doc__ = f.__doc__
    return newf

And on decorated function add:

@checkStale
def open(self):
    """
    open()
    Some docs.
    """
    # Code

Answer 6

If you're particularly adamant about not adding another dependency here's a code snippet that works with the regular inspector by injecting into the docstring. It's quite hackey and not really recommended unless there are good reasons to not add another module, but here it is.

# inject the wrapped functions signature at the top of a docstring
args, varargs, varkw, defaults = inspect.getargspec(method)
defaults = () if defaults is None else defaults
defaults = ["\"{}\"".format(a) if type(a) == str else a for a in defaults]
l = ["{}={}".format(arg, defaults[(idx+1)*-1]) if len(defaults)-1 >= idx else arg for idx, arg in enumerate(reversed(list(args)))]
if varargs: allargs.append('*' + varargs)
if varkw: allargs.append('**' + varkw)
doc = "{}({})\n{}".format(method.__name__, ', '.join(reversed(l)), method.__doc__)
wrapper.__doc__ = doc

Answer 7

the answer to this is quite simple, but none of the threads I've seen have mentioned it. Have a look at functools.update_wrapper()

import functools

def schema_in(orig_func):
    schema = Schema() 
    def validate_args(*args, **kwargs):
        clean_kwargs = schema.load(**kwargs)
        return orig_func(**clean_kwargs)

    functools.update_wrapper(validate_args, orig_func)
    return validate_args
    

I'm not sure this will run, but it illustrates the concept. If your wrapper is injecting validated_args between the caller and the callee, the example shows how to update the wrapper (validated_args) method with the metadata of orig_method. Ultimately, this will allow Sphinx and other type analysis tools such as mypy (I'm assuming!) to see the data needed to behave as expected. I have just finished testing this and can confirm it works as described, Sphinx autodoc is behaving as desired.

Answer 8

In recent versions of python, you can update the signature of the decorated function from the decorator itself. For example:

import inspect

def my_decorator(f):
    def new_f(*args, **kwargs):
        # Decorate function
        pass

    new_f.__doc__ = f.__doc__
    new_f.__module__ = f.__module__
    new_f.__signature__ = inspect.signature(f)
    return new_f

Answer 9

UPDATE: this may be "impossible" to do cleanly because sphinx uses the function's code object to generate its function signature. But, since you're using sphinx, there is a hacky workaround that does works.

It's hacky because it effectively disables the decorator while sphinx is running, but it does work, so it's a practical solution.

At first I went down the route of constructing a new types.CodeType object, to replace the wrapper's func_code code object member, which is what sphinx uses when generating the signatures.

I was able to segfault python by going down the route or trying to swap in the co_varnames, co_nlocals, etc. members of the code object from the original function, and while appealing, it was too complicated.

The following solution, while it is a hacky heavy hammer, is also very simple =)

The approach is as follows: when running inside sphinx, set an environment variable that the decorator can check. inside the decorator, when sphinx is detected, don't do any decorating at all, and instead return the original function.

Inside your sphinx conf.py:

import os
os.environ['SPHINX_BUILD'] = '1'

And then here is an example module with a test case that shows what it might look like:

import functools
import os
import types
import unittest


SPHINX_BUILD = bool(os.environ.get('SPHINX_BUILD', ''))


class StaleError(StandardError):
    """Custom exception for staleness"""
    pass


def check_stale(f):
    """Raise StaleError when the object has gone stale"""

    if SPHINX_BUILD:
        # sphinx hack: use the original function when sphinx is running so that the
        # documentation ends up with the correct function signatures.
        # See 'SPHINX_BUILD' in conf.py.
        return f

    @functools.wraps(f)
    def wrapper(self, *args, **kwargs):
        if self.stale:
            raise StaleError('stale')

        return f(self, *args, **kwargs)
    return wrapper


class Example(object):

    def __init__(self):
        self.stale = False
        self.value = 0

    @check_stale
    def get(self):
        """docstring"""
        return self.value

    @check_stale
    def calculate(self, a, b, c):
        """docstring"""
        return self.value + a + b + c


class TestCase(unittest.TestCase):

    def test_example(self):

        example = Example()
        self.assertEqual(example.get(), 0)

        example.value = 1
        example.stale = True
        self.assertRaises(StaleError, example.get)

        example.stale = False
        self.assertEqual(example.calculate(1, 1, 1), 4)


if __name__ == '__main__':
    unittest.main()