29

I'm trying to add cross-references to external API into my documentation but I'm facing three different behaviors.

I am using sphinx(1.3.1) with Python(2.7.3) and my intersphinx mapping is configured as:

{
'python': ('https://docs.python.org/2.7', None),
'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'cv2' : ('http://docs.opencv.org/2.4/', None),
'h5py' : ('http://docs.h5py.org/en/latest/', None)
}

I have no trouble writing a cross-reference to numpy API with :class:`numpy.ndarray` or :func:`numpy.array` which gives me, as expected, something like numpy.ndarray.

However, with h5py, the only way I can have a link generated is if I omit the module name. For example, :class:`Group` (or :class:`h5py:Group`) gives me Group but :class:`h5py.Group` fails to generate a link.

Finally, I cannot find a way to write a working cross-reference to OpenCV API, none of these seems to work:

:func:`cv2.convertScaleAbs`
:func:`cv2:cv2.convertScaleAbs`
:func:`cv2:convertScaleAbs`
:func:`convertScaleAbs`

How to properly write cross-references to external API, or configure intersphinx, to have a generated link as in the numpy case?

Gall
  • 1,595
  • 1
  • 14
  • 22

7 Answers7

30

In addition to the detailed answer from @gall, I've discovered that intersphinx can also be run as a module:

python -m sphinx.ext.intersphinx 'http://python-eve.org/objects.inv'

This outputs nicely formatted info. For reference: https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/intersphinx.py#L390

lingfish
  • 422
  • 5
  • 6
  • 2
    Broken in Sphinx 1.4.3? I just get `intersphinx inventory 'http://docs.python.org/' not readable due to AttributeError: 'MockApp' object has no attribute 'info'` – akaihola Jun 09 '16 at 20:08
  • 3
    It does work on a local file: `wget http://python-eve.org/objects.inv`, then `python -m sphinx.ext.intersphinx object.inv` – Brecht Machiels Aug 01 '16 at 11:59
  • 3
    `python -m sphinx.ext.intersphinx 'http://python-eve.org/objects.inv'` works again in Sphinx 1.6.2. – mzjn Jul 15 '17 at 09:46
  • 1
    And it [stopped working in 1.7.0](https://github.com/sphinx-doc/sphinx/issues/4664). – ederag Feb 22 '18 at 16:05
27

I gave another try on trying to understand the content of an objects.inv file and hopefully this time I inspected numpy and h5py instead of only OpenCV's one.

How to read an intersphinx inventory file

Despite the fact that I couldn't find anything useful about reading the content of an object.inv file, it is actually very simple with the intersphinx module.

from sphinx.ext import intersphinx
import warnings


def fetch_inventory(uri):
    """Read a Sphinx inventory file into a dictionary."""
    class MockConfig(object):
        intersphinx_timeout = None  # type: int
        tls_verify = False

    class MockApp(object):
        srcdir = ''
        config = MockConfig()

        def warn(self, msg):
            warnings.warn(msg)

    return intersphinx.fetch_inventory(MockApp(), '', uri)


uri = 'http://docs.python.org/2.7/objects.inv'

# Read inventory into a dictionary
inv = fetch_inventory(uri)
# Or just print it
intersphinx.debug(['', uri])

File structure (numpy)

After inspecting numpy's one, you can see that keys are domains:

[u'np-c:function',
 u'std:label',
 u'c:member',
 u'np:classmethod',
 u'np:data',
 u'py:class',
 u'np-c:member',
 u'c:var',
 u'np:class',
 u'np:function',
 u'py:module',
 u'np-c:macro',
 u'np:exception',
 u'py:method',
 u'np:method',
 u'np-c:var',
 u'py:exception',
 u'np:staticmethod',
 u'py:staticmethod',
 u'c:type',
 u'np-c:type',
 u'c:macro',
 u'c:function',
 u'np:module',
 u'py:data',
 u'np:attribute',
 u'std:term',
 u'py:function',
 u'py:classmethod',
 u'py:attribute']

You can see how you can write your cross-reference when you look at the content of a specific domain. For example, py:class:

{u'numpy.DataSource': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.DataSource.html#numpy.DataSource',
  u'-'),
 u'numpy.MachAr': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.MachAr.html#numpy.MachAr',
  u'-'),
 u'numpy.broadcast': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html#numpy.broadcast',
  u'-'),
  ...}

So here, :class:`numpy.DataSource` will work as expected.

h5py

In the case of h5py, the domains are:

[u'py:attribute', u'std:label', u'py:method', u'py:function', u'py:class']

and if you look at the py:class domain:

{u'AttributeManager': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/attr.html#AttributeManager',
  u'-'),
 u'Dataset': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/dataset.html#Dataset',
  u'-'),
 u'ExternalLink': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/group.html#ExternalLink',
  u'-'),
 ...}

That's why I couldn't make it work as numpy references. So a good way to format them would be :class:`h5py:Dataset`.

OpenCV

OpenCV's inventory object seems malformed. Where I would expect to find domains there is actually 902 function signatures:

[u':',
 u'AdjusterAdapter::create(const',
 u'AdjusterAdapter::good()',
 u'AdjusterAdapter::tooFew(int',
 u'AdjusterAdapter::tooMany(int',
 u'Algorithm::create(const',
 u'Algorithm::getList(vector<string>&',
 u'Algorithm::name()',
 u'Algorithm::read(const',
 u'Algorithm::set(const'
 ...]

and if we take the first one's value:

{u'Ptr<AdjusterAdapter>': (u'OpenCV',
  u'2.4',
  u'http://docs.opencv.org/2.4/detectorType)',
  u'ocv:function 1 modules/features2d/doc/common_interfaces_of_feature_detectors.html#$ -')}

I'm pretty sure it is then impossible to write OpenCV cross-references with this file...

Conclusion

I thought intersphinx generated the objects.inv based on the content of the documentation project in an standard way, which seems not to be the case. As a result, it seems that the proper way to write cross-references is API dependent and one should inspect a specific inventory object to actually see what's available.

Gall
  • 1,595
  • 1
  • 14
  • 22
  • 2
    side note: opencv3 has removed all sphinx documentation (in favour of doxygen) – berak Jun 22 '15 at 13:51
  • I'm trying to get a link to np.float_ and it's not working. I'm trying things like `:c:var:\`NPY_FLOAT\`` but sphinx complains of an 'unknown text role'. If I use `:class:\`NPY_FLOAT\`` intersphinx doesn't lay in the link properly. Any thoughts? (Nice work here, btw!) – hBy2Py Nov 21 '15 at 13:23
  • @Brian As I understand the Sphinx documentation, *Domains* are used to describe something and *Cross-references* are basically links to those resources. If you look at the [C domain](http://sphinx-doc.org/domains.html#the-c-domain), you'll see that ``var`` is available as a *domain* (to describe a variable) but not as a *cross-reference*. In this case you have to use ``:c:data:`NPY_FLOAT` ``. – Gall Nov 23 '15 at 08:39
  • Mmph, `:c:data:` might be the one I didn't try. Pretty sure I tried `:c:attr:`, at least, and it didn't work. I'll give `data` a shot... thanks! – hBy2Py Nov 23 '15 at 11:51
  • 3
    Instead of writing this Python program which does not work anymore, the information can be extracted through running the intersphinx module as per [this other answer](https://stackoverflow.com/a/36831198/1735594): python -m sphinx.ext.intersphinx FILE.inv – user1735594 Feb 28 '18 at 16:30
  • There's a guide to the syntax of the inventory here: in particular the format of each line of object data goes `{name} {domain}:{role} {priority} {uri} {dispname}` https://buildmedia.readthedocs.org/media/pdf/sphobjinv/v.doc/sphobjinv.pdf – Louis Maddox Oct 03 '22 at 17:34
5

An additional way to inspect the objects.inv file is with the sphobjinv module.

You can search local or even remote inventory files (with fuzzy matching). For instance with scipy:

$ sphobjinv suggest -t 90 -u https://docs.scipy.org/doc/scipy/reference/objects.inv "signal.convolve2d"

Remote inventory found.

:py:function:`scipy.signal.convolve2d`
:std:doc:`generated/scipy.signal.convolve2d`

Note that you may need to use :py:func: and not :py:function: (I'd be happy to know why).

Francis Colas
  • 3,459
  • 2
  • 26
  • 31
  • 6
    Having to use `:func:` in cross-references instead of the `:function:` listed in the `objects.inv` is just an idiosyncrasy of the Python domain in Sphinx. All of this terminology is defined [here](https://github.com/sphinx-doc/sphinx/blob/685e3fdb49c42b464e09ec955e1033e2a8729fff/sphinx/domains/python.py#L845-L881); you'd have to consult the docs for the underlying `docutils` to learn what all of the syntax is actually doing. – hBy2Py Oct 15 '19 at 04:47
  • 1
    Here is also the relevant section of the documentation on how to reference Python objects: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#python-roles – moi Oct 20 '19 at 09:01
3

How to use OpenCV 2.4 (cv2) intersphinx

Inspired by @Gall's answer, I wanted to compare the contents of the OpenCV & numpy inventory files. I couldn't get sphinx.ext.intersphinx.fetch_inventory to work from ipython, but the following does work:

curl http://docs.opencv.org/2.4/objects.inv | tail -n +5 | zlib-flate -uncompress > cv2.inv
curl https://docs.scipy.org/doc/numpy/objects.inv | tail -n +5 | zlib-flate -uncompress > numpy.inv

numpy.inv has lines like this:

numpy.ndarray py:class 1 reference/generated/numpy.ndarray.html#$ -

whereas cv2.inv has lines like this:

cv2.imread ocv:pyfunction 1 modules/highgui/doc/reading_and_writing_images_and_video.html#$ -

So presumably you'd link to the OpenCV docs with :ocv:pyfunction:`cv2.imread` instead of :py:function:`cv2.imread`. Sphinx doesn't like it though:

WARNING: Unknown interpreted text role "ocv:pyfunction".

A bit of Googling revealed that the OpenCV project has its own "ocv" sphinx domain: https://github.com/opencv/opencv/blob/2.4/doc/ocv.py -- presumably because they need to document C, C++ and Python APIs all at the same time.

To use it, save ocv.py next to your Sphinx conf.py, and modify your conf.py:

sys.path.insert(0, os.path.abspath('.'))
import ocv
extensions = [
    'ocv',
]
intersphinx_mapping = {
    'cv2': ('http://docs.opencv.org/2.4/', None),
}

In your rst files you need to say :ocv:pyfunc:`cv2.imread` (not :ocv:pyfunction:).

Sphinx prints some warnings (unparseable C++ definition: u'cv2.imread') but the generated html documentation actually looks ok with a link to http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#cv2.imread. You can edit ocv.py and remove the line that prints that warning.

David Röthlisberger
  • 1,786
  • 15
  • 20
2

The accepted answer no longer works in the new version (1.5.x) ...

import requests
import posixpath
from sphinx.ext.intersphinx import read_inventory

uri = 'http://docs.python.org/2.7/'

r = requests.get(uri + 'objects.inv', stream=True)
r.raise_for_status()

inv = read_inventory(r.raw, uri, posixpath.join)
Eric
  • 95,302
  • 53
  • 242
  • 374
  • This answer does not work either (tested with 1.6.2): "ImportError: cannot import name read_inventory". – mzjn Jul 15 '17 at 08:33
  • I was using 1.5.x at the time, so you're probably right. Want to submit an answer that works in 1.6.2? – Eric Jul 15 '17 at 08:38
  • The answer from @lingfish stopped working in 1.4.3. But it works again in 1.6.2. – mzjn Jul 15 '17 at 09:49
2

Stubborn fool that I am, I used 2to3 and the Sphinx deprecated APIs chart to revive @david-röthlisberger's ocv.py-based answer so it'll work with Sphinx 2.3 on Python 3.5.

The fixed-up version is here:

https://gist.github.com/ssokolow/a230b27b7ea4a31f7fb40621e6461f9a

...and the quick version of what I did was:

  1. Run 2to3 -w ocv.py && rm ocv.py.bak
  2. Cycle back and forth between running Sphinx and renaming functions to their replacements in the chart. I believe these were the only changes I had to make on this step:
    1. Directive now has to be imported from docutils.parsers.rst
    2. Replace calls to l_(...) with calls to _(...) and remove the l_ import.
  3. Replace calls to env.warn with calls to log.warn where log = sphinx.util.logging.getLogger(__name__).

Then, you just pair it with this intersphinx definition and you get something still new enough to be relevant for most use cases:

'cv2': ('https://docs.opencv.org/3.0-last-rst/', None)
ssokolow
  • 14,938
  • 7
  • 52
  • 57
1

For convenience, I made a small extension for aliasing intersphinx cross references. This is useful as sometimes the object inventory gets confused when an object from a submodule is imported from a package's __init__.py.

See also https://github.com/sphinx-doc/sphinx/issues/5603

###
# Workaround of
# Intersphinx references to objects imported at package level can"t be mapped.
#
# See https://github.com/sphinx-doc/sphinx/issues/5603

intersphinx_aliases = {
    ("py:class", "click.core.Group"):
        ("py:class", "click.Group"),
    ("py:class", "click.core.Command"):
        ("py:class", "click.Command"),
}


def add_intersphinx_aliases_to_inv(app):
    from sphinx.ext.intersphinx import InventoryAdapter
    inventories = InventoryAdapter(app.builder.env)

    for alias, target in app.config.intersphinx_aliases.items():
        alias_domain, alias_name = alias
        target_domain, target_name = target
        try:
            found = inventories.main_inventory[target_domain][target_name]
            try:
                inventories.main_inventory[alias_domain][alias_name] = found
            except KeyError:
                print("could not add to inv")
                continue
        except KeyError:
            print("missed :(")
            continue


def setup(app):
    app.add_config_value("intersphinx_aliases", {}, "env")
    app.connect("builder-inited", add_intersphinx_aliases_to_inv)

To use this, I paste the above code in my conf.py and add aliases to the intersphinx_aliases dictionary.

abstrus
  • 1,143
  • 7
  • 8