5

Problem

I'm trying to build documentation for a Python project that contains a number of subdirectories. I'm trying to use sphinx-apidoc to make the process seamless. However, despite my best efforts, I can't deal with code that is in subdirectories. I followed a great short tutorial on YouTube here and it worked well. To mock the issue that I've been having, I put one of the files in a can_it_handle_folders directory, as below.

spamfilter-py
│   __init__.py  (**)
│   readme.md
│   sample.py
│   spamfilter.py
│   token.py
├───can_it_handle_folders
│       __init__.py
│       test_spamfilter.py
├───docs
│   ├───build
│   │   └───html
│   └───source
│       ├───conf.py
│       ├───index.rst
│       ├───modules.rst
│       ├───sample.rst
│       ├───spamfilter.rst
│       ├───token.rst
│       ├───spamfilter-py.test_spamfilter.rst
│       └───html
...

I go to the docs directory and run sphinx-apidoc -o . .. to generate .rst files based on the root spamfilter directory. I've added the following line to the conf.py:

sys.path.insert(0, os.path.abspath('..'))

and my generated modules.rst looks like:

spamfilter-py
=============

.. toctree::
   :maxdepth: 4

   sample
   spamfilter
   token

The html for the anything in the can_it_handle_folders directory does not generate. If I try to add can_it_handle_folders/test_spamfilter to the toctree, I get a toctree contains reference to nonexisting document 'can_it_handle_folders/test_spamfilter' error.

Question

I want the test_spamfilter module to show up in the generated html. How do I do this? What is the best way to set this up?

UPDATE: it is the init file

I've isolated the issue, which seems to be the __init__.py in the root directory (marked with ** above). I don't know how to deal with this. If I remove it, sphinx works exactly how I want it to. If I keep it, the issue becomes this one here. It seems that this must be an issue with the sys.path. This is the error I get, for each of the files:

WARNING: invalid signature for automodule ('spamfilter-py.can_it_handle_folders')
WARNING: don't know which module to import for autodocumenting 'spamfilter-py.can_it_handle_folders' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)

I have tried the following in the conf.py file:

  • sys.path.insert(0, os.path.abspath(os.path.join('..')))
  • sys.path.insert(0, os.path.abspath(os.path.join('..', '..')))
  • sys.path.insert(0, os.path.abspath(os.path.join('..', '..', '..')))

All have resulted in the same error message above. I have also tried running sphinx-apidoc -o ./source ../.. in addition to sphinx-apidoc -o ./source .. for each of the paths above. What could be the issue? I know it must be some small configuration thing.

Context

I am an absolute beginner to Sphinx. I have tried to read the documentation, but it has not addressed this issue. I've tried a number of things that I guess are clearly incorrect. If there is no easy answer, I'll add my attempts here. I know that there are other stack overflow questions on this, but they are not recent and unhelpful.

  • Windows 10 computer using PowerShell for commands

Included files:

  • index.rst
Welcome to j's documentation!
=============================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
  • conf.py
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join('..', '..')))


# -- Project information -----------------------------------------------------

project = 'j'
copyright = '2020, k'
author = 'k'


# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc'
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
  • one rst that is not working, spamfilter-py.rst:
spamfilter\-py package
======================

Subpackages
-----------

.. toctree::
   :maxdepth: 4

   spamfilter-py.can_it_handle_folders

Submodules
----------

spamfilter\-py.sample module
----------------------------

.. automodule:: spamfilter-py.sample
   :members:
   :undoc-members:
   :show-inheritance:

spamfilter\-py.spamfilter module
--------------------------------

.. automodule:: spamfilter-py.spamfilter
   :members:
   :undoc-members:
   :show-inheritance:

spamfilter\-py.token module
---------------------------

.. automodule:: spamfilter-py.token
   :members:
   :undoc-members:
   :show-inheritance:

Module contents
---------------

.. automodule:: spamfilter-py
   :members:
   :undoc-members:
   :show-inheritance:

Here is what the documentation looks like: No methods or parameters

whoopscheckmate
  • 746
  • 8
  • 23
  • 3
    The problem as you describe it has several steps that should be taken for things to work. First step that is generally recommended is choosing the option "seperate sources from build" when running sphinx-quickstart. (If you don't your project will be a mess). Afterwards both your sphinx-apidoc and conf.py will be written differently because the source and destination folders will be different. Your packages (with .py files) should have an `__init__.py` otherwise Python can't import the modules. Carefully follow [these steps](https://stackoverflow.com/a/59951675) let us know how it goes. – bad_coder Sep 28 '20 at 17:38
  • 1
    Ok. So I went through all those instructions and the same issue happened. Then I added a ```__init__.py``` for each subdirectory. Now it writes an rst for all the modules (yay!), however, I get this warning for all of them ```WARNING: Unknown directive type "automodule".``` The documentation for each is missing. What do I need to do now? – whoopscheckmate Sep 28 '20 at 18:55
  • Ahh, you have to add 'sphinx.ext.autodoc' to the list of extensions in ```conf.py```. Perfect. It's now working. Thank you for the help @bad_coder – whoopscheckmate Sep 28 '20 at 19:00
  • It would have been advisable to start with a project having 1 single .rst and .py file to make things easier. With the amount of files you have problems are more difficult to diagnose. The warning you are getting means the automodule in the generated .rst files isn't being recognized, the reason for that is you didn't include the [autodoc](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html) extension in your conf.py, the line where it says`extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.napoleon' ]` – bad_coder Sep 28 '20 at 19:02
  • 2
    Of course! Thank you. I did start with a small project, but making the leap to directories confused me, clearly. – whoopscheckmate Sep 28 '20 at 19:14
  • Now I'm having an issue where it doesn't document any of the methods in the modules. It has the right hierarchy, but "Module contents" is empty. Is there an obvious reason for this? – whoopscheckmate Sep 28 '20 at 19:22
  • Ok. The modules.rst is linked to the the index.rst, like in your instructions. The problem I'm having [sounds like this](https://stackoverflow.com/questions/38885106/sphinx-apidoc-picks-up-submodules-but-autodoc-doesnt-document-them). There are docstrings, but for each automodule, it includes ```:members:``` and ```:undoc-members:``` anyways. – whoopscheckmate Sep 28 '20 at 20:37
  • In VScode I'm getting a lint warning that could be helpful. ```No directive entry for "automodule" in module "docutils.parsers.rst.languages.en".restructuredtext``` Is this meaningful? – whoopscheckmate Sep 28 '20 at 20:39
  • 1
    At this point the question would need an overhaul since it's unclear what your Sphinx is doing. Include the `conf.py` include `index.rst` and `modules.rst` and include 1 example `.rst` file you think isn't working together with the corresponding `.py`. Also, refresh you file structure to show the `__init__.py`. If the Python docstrings are getting picked up and rendered as HTML you are good! I don't use VScode and so far PyCharm also doesn't process `.rst` files correctly, there are some custom linters out there but I have doubts they work. Criteria is your docstrings getting turned into HTML. – bad_coder Sep 28 '20 at 20:54
  • Your linter is in fact wrong and you should ignore that warning. Docutils doesn't process autodoc, it's an independent directive, that's what the warning is saying. – bad_coder Sep 28 '20 at 20:56
  • @bad_coder, I've made the update, and once I add an __init__.py in the root, the problem occurs. – whoopscheckmate Sep 29 '20 at 13:47
  • 1
    Your directory/file structure doesn't have a `/source` folder that you are passing as an argument to sphinx-apidoc. You should run sphinx-quickstart and choose "separate sources from build" as I indicated in the first comment. – bad_coder Sep 29 '20 at 15:36
  • @bad_coder thanks for the reminder to update the other things. I should have most of it up there. – whoopscheckmate Sep 29 '20 at 20:04
  • With this exact configuration try adding an extra pair of dots in the conf.py where you set the sys.path. Explanation being: the dots are the relative path taken from where conf.py is. It's in docs/source, so 2 levels down places you in the root package. Problem is, for the import of a package to work you aren't supposed to be inside it, but one level below it. That's why the `__init__.py` in the root of the project makes things stop working. (This sort of stuff is hard to diagnose with a changing problem that only gives incomplete info...) – bad_coder Sep 29 '20 at 20:27
  • I think the warning you are getting "spamfilter-py.can_it_handle_folders" is about the `.rst` file. The entries in the toctree are `.rst` files not `.py` modules. So if you look at the dirs/files you are showing there's no `.rst` file with that name present...Best approach in this case is: Backup you conf.py and index.rst, delete everything else inside `docs` and generate the `.rst` files from scratch using sphinx-apidoc so you're certain the `.rst` files correspond to you current package/module layout...(Notice sphinx-apidoc is a convenience, after a while you write `.rst` by hand.) – bad_coder Sep 29 '20 at 20:41
  • Does this answer your question? [Sphinx's autodoc's automodule having apparently no effect](https://stackoverflow.com/questions/59903051/sphinxs-autodocs-automodule-having-apparently-no-effect) – Daniel Widdis Oct 01 '20 at 03:49

0 Answers0