Sphinx "autodoc" extension unable to even find standard modules guaranteed to exist

Question

I'm attempting to document the public API of beartype, a berry fun runtime type checker for Python. I am failing utterly. Sphinx's autodoc extension fails to find standard pure-Python CPython modules that are both guaranteed to exist and be trivially importable. This should not be... yet it is.

This question is thus divergent from every other question on the topic. As the print() statement I've manually injected into the doc/source/conf.py Sphinx configuration for this project demonstrates, Sphinx's runtime sys.path is sanely established and should yield worky.

In defiance of sanity, autodoc fails to resolve cross references to even standard modules:

$ sphinx-build -M html doc/source/ doc/build/ -W -j auto --keep-going -n -a
Running Sphinx v4.4.0
sys.path (from "doc/source/conf.py"): ['/usr/lib/python-exec/python3.8', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/lib/python3.8/site-packages', '/home/leycec/py/beartype']
loading pickled environment... done
[autosummary] generating autosummary for: changes.rst, index.rst, reference.rst, tutorial.rst
building [mo]: all of 0 po files
building [html]: all source files
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
preparing documents... done
/usr/lib/python3.8/site-packages/beartype/__init__.py:docstring of beartype:3: WARNING: py:mod reference target not found: beartype.meta
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: beartype._data.datatyping.BeartypeableT
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: beartype._data.datatyping.BeartypeableT
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: beartype._data.datatyping.BeartypeableT
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: BeartypeableT
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: BeartypeConf
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: optional
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:class reference target not found: BeartypeReturn
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeConfException
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:68: WARNING: py:class reference target not found: BeartypeConf
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeDecorHintException
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:69: WARNING: py:mod reference target not found: typing
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeDecorHintPep563Exception
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeDecorParamNameException
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeDecorWrappeeException
/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:: WARNING: py:exc reference target not found: BeartypeDecorWrapperException

waiting for workers...
generating indices... genindex py-modindex done
highlighting module code... [100%] beartype._decor.cache.cachedecor                                                            
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build finished with problems, 16 warnings.

Ignore everything above except this warning on autodoc failing to find the standard typing module:

/usr/lib/python3.8/site-packages/beartype/_decor/cache/cachedecor.py:docstring of beartype._decor.cache.cachedecor.beartype:69: WARNING: py:mod reference target not found: typing

"/usr/lib/python3.8" resides in sys.path. "/usr/lib/python3.8/typing.py" exists, as indeed it must. Yet autodoc fails to find anything. Insert double facepalm meme here.

I'm increasingly under the suspicion that autodoc is Hell Incarnate. Most of us know this already. The rest are able to forcefully bend Sphinx to their perfidious will through arcane magic beyond my feeble talents. I'm in the former category.

Tangentially: Are there any viable alternatives to autodoc that I can immediately bludgeon into a semblance of working short of manually documenting everything in raw Sphinx-specific reStructuredText files (which isn't going to happen, because there are not enough free unpaid volunteer seconds remaining in the lifetime of the known universe for that)?

Because... this isn't working.

Could you please reduce your problem to a minimal example of documentation that you want to process through Sphinx? E.g., one that only uses `typing`, and not a complete package. — 9769953, Mar 26 '22 at 08:21
Sphinx does not fail to import `typing` or other standard modules. All the warnings are about missing cross-reference targets. In order to get working cross-references to standard modules, use intersphinx (https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html). I think the other warnings are there because you have cross-references to private modules in Beartype for which no HTML output is being generated. — mzjn, Mar 26 '22 at 09:28
`intersphinx`: so, this is a thing. Thanks for the heads up, @mzjn! You're awesome and you know it. I foolishly assumed cross-referencing to standard modules was builtin (because *it absolutely should be*) rather than requiring yet another extension to be configured. For sanity, I'll self-answer with your commentary as the content. — Cecil Curry, Mar 30 '22 at 05:37

score 1 · Answer 1 · answered Mar 30 '22 at 05:43

Courtesy Sphinx wizard @mzjn as an astute comment above:

Sphinx does not fail to import typing or other standard modules. All the warnings are about missing cross-reference targets. In order to get working cross-references to standard modules, use intersphinx.

I foolishly assumed cross-referencing to standard modules was builtin... because it absolutely should be. Sadly, doing so requires yet another orthogonal extension to be configured. Naturally, Sphinx fails to suggest this might be a good idea in its warning output – which Sphinx doesn't even emit unless you explicitly pass the secretive -n option to sphinx-build. </double_facepalm>

Thanks a metric ton, @mzjn. You're awesome and you know it.

FYI, [this is the canonical SO thread](https://stackoverflow.com/a/30981554/) and I do prefer using `objects.inv` while developing on my localhost. While you'd expect Sphinx to warn you fact of the matter is the devs have this issue well stated in the Sphinx documentation (Sphinx isn't the kind of beast where you trust the compiler over the docs or GitHub). I'm surprised the Sphinx devs manage to get so much good work done as it is - there's thousands of users but just a handful of contributors... — bad_coder, Apr 19 '22 at 07:44

Sphinx "autodoc" extension unable to even find standard modules guaranteed to exist

1 Answers1