Using sphinx autodoc for a fabfile

Question

Is it possible to use Sphinx autodoc to generate documentation for my fabfile, from the function docstrings?

E.g. for a fabfile containing a setup_development task i've tried:

.. automodule::fabfile
   :members:
   .. autofunction:: setup_development

But nothing is generated.

fabfile snippet:

@task
def setup_development(remote='origin', branch='development'):
    """Setup your development environment.

    * Checkout development branch & pull updates from remote
    * Install required python packages
    * Symlink development settings
    * Sync and migrate database
    * Build HTML Documentation and open in web browser

    Args:
        remote: Name of remote git repository. Default: 'origin'.
        branch: Name of your development branch. Default: 'development'.
    """
    <code>

One problem is the missing space between `.. automodule::` and `fabfile`. And if you want to use `..autofunction::` (I don't think you need it), it should be preceded by a blank line. — mzjn, Jan 13 '12 at 18:32
Thanks, that did the trick! Looks like ``:members:`` does not pick up wrapped functions (as per answer given by shahjapan), so I can either use ``wraps`` or use ``.. autofunction::`` which works with the ``@task`` decorator. — Matt Austin, Jan 14 '12 at 00:50

score 3 · Answer 1 · edited Apr 03 '20 at 17:47

3

Its because you've applied decorator on your function setup_development

you need to update your task function with functools.wraps as below,

from functools import wraps

def task(calling_func):
    @wraps(calling_func)
    def wrapper_func(self, *args, **kw):
        return calling_func(*args, **kw)
    return wrapper_func

If you document decorated functions or methods, keep in mind that autodoc retrieves its docstrings by importing the module and inspecting the __doc__ attribute of the given function or method.

That means that if a decorator replaces the decorated function with another, it must copy the original __doc__ to the new function. From Python 2.5, functools.wraps() can be used to create well-behaved decorating functions.

References:

edited Apr 03 '20 at 17:47

bad_coder

11,289
20
44
72

answered Jan 13 '12 at 05:01

shahjapan

13,637
22
74
104

I can't for the life of me get this to work with the existing ``fabric.api.task`` decorator. Any change you could provide the specific code to decorate this existing decorator to pass-through ``__doc__``? Thanks. – Matt Austin Jan 16 '12 at 00:23
@Matt Austin: So `autofunction` doesn't work with `@task` after all? – mzjn Jan 20 '12 at 16:20
@mzjn autofunction half-works, the docstring information is generated, but the function args/kwargs are missing. – Matt Austin Jan 21 '12 at 02:57
1

@shahjapan, can you explain why your decorator function looks the way it does? What is `calling_wrapper`? Where does `perm_context` come from? – mzjn Jan 21 '12 at 08:05
1

@shahjapan, I still feel there's something missing. You say to Matt: "you need to update your `task` function with `functools.wraps` as below". How exactly is that supposed to be done? The original `task` function comes from the [Fabric library](http://docs.fabfile.org/en/1.3.3/usage/tasks.html#the-task-decorator). – mzjn Jan 23 '12 at 16:59

mzjn · Accepted Answer · 2013-01-17T18:47:59.117

I was able to produce full documentation with preserved function signature by using the decorator_apply recipe found in the documentation for the decorator module.

""" myfabfile.py """

from fabric.api import task as origtask
from decorator import FunctionMaker

def decorator_apply(dec, func):
    return FunctionMaker.create(
        func, 'return decorated(%(signature)s)',
        dict(decorated=dec(func)), __wrapped__=func)

def task(func):
    return decorator_apply(origtask, func)

@task
def setup_development(remote='origin', branch='development'):
    """Setup your development environment.

    * Checkout development branch & pull updates from remote
    * Install required python packages
    * Symlink development settings
    * Sync and migrate database
    * Build HTML Documentation and open in web browser

    :param remote: Name of remote git repository.
    :param branch: Name of your development branch.
    """
    pass

This is the simple ReST source that I used:

.. automodule:: myfabfile
   :members:

Some comments:

The answer submitted by shahjapan explains how to preserve the docstring in the general case, but it does not address the fact that the @task decorator is defined in an external library.

Anyway, it turns out that the docstring is preserved automatically for functions decorated with @task. The following is in the __init__ method of Fabric's tasks.WrappedCallableTask class:

if hasattr(callable, '__doc__'):
    self.__doc__ = callable.__doc__

So that already works as it is (an explicit .. autofunction:: is needed). To ensure that the function signature is preserved as well, the decorator module can be used as shown above.

Update

The use of the decorator module breaks things in the workings of Fabric (see comment). So that may not be feasible after all. As an alternative I suggest the following modified reST markup:

.. automodule:: myfabfile2
   :members: 

   .. autofunction:: setup_development(remote='origin', branch='development')

That is, you'll have to include the full function signature. This is also what is suggested in the Sphinx documentation (see "This is useful if the signature from the method is hidden by a decorator.").

This works for autodoc, but using this method fabric no longer recognises its decorated tasks ``fab --list`` lists all functions (this is the fallback behaviour for backwards compatibility), in which case it would be easier to just omit ``@task`` completely. It also prevents imported tasks being found in other files when using ``fabfile`` as a module. — Matt Austin, Feb 20 '12 at 03:05
I guess the only way then is to duplicate the function names and signatures in both files. A bit of a pain, but thanks for investigating. — Matt Austin, Feb 26 '12 at 01:14
As we haven't really found a way to automatically generate docs for a fabfile, I've created an issue for Fabric over at github: https://github.com/fabric/fabric/issues/569 — Matt Austin, Feb 26 '12 at 02:53
Matt, in the issue report, you label both workarounds as "unsuccessful". But I'd say that manually specifying the function signature in the .rst file is a reasonable compromise. Yes, it is a bit of a pain that violates the "don't repeat yourself" principle. But both the Sphinx documentation that I referred to and the comment from the Fabric maintainer indicate that there is no better solution (for now at least). @task turns out to be especially complicated since it "involves a wrapped/delegate class and not just a nested or decorated function object". — mzjn, Feb 26 '12 at 08:27
Yes, unfortunately I'll have to rely on remembering to update the sphinx signatures whenever I add/modify a task. At least the documentation is still generated this way, and it's a lot better than what I had at the beginning of this issue. Thanks for your help. — Matt Austin, Feb 26 '12 at 12:55
Thank you! It was fun investigating this issue (and I learned a thing or two along the way). — mzjn, Feb 26 '12 at 14:36

Using sphinx autodoc for a fabfile

2 Answers2

Linked