Building docs fails due to missing pandoc

Question

The problem

I am having trouble getting my docs to build successfully when clicking "Build" on the readthedocs.io web interface, but it builds just fine on my local machine. To test that it is an environment issue, I created a virtual environment:

conda create virtualenv -n venv
conda env export -n venv
source activate venv

Then I installed my requirements.txt file as:

pip install -r requirements.txt

and then ran

make clean html

In the virtual env and on buildthedocs online, I get the error:

Notebook error:
PandocMissing in ex_degassing.ipynb:
Pandoc wasn't found.
Please check that pandoc is installed:
http://pandoc.org/installing.html
make: *** [html] Error 2

I have searched and searched for a solution, but my best guess is that pandoc is not being installed via pip even though it is in the requirements.txt file. I tried also telling it to build from source by replacing pandoc with git+git://github.com/jgm/pandoc#egg=pandoc in my requirements.txt file, but this did not work (see below for what my files look like). I can easily install pandoc on my local machine, but it fails to install via the requirements.txt file in my virtual environment or on readthedocs.

Some of my files

Here is my requirements.txt file:

sphinx>=1.4
sphinx_rtd_theme
ipykernel
nbsphinx
pandas
pandoc
numpy
matplotlib
ipynb
scipy

I read through all of the readthedocs documentation and, ironically, found the documentation for building woefully inadequate for someone of my limited coding ability. I tried making both readthedocs.yml and environment.yaml files, but I have no idea if they are doing anything or if they are properly written. For completeness, here are those files:

readthedocs.yml

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2
# Build documentation in the docs/ directory with Sphinx
sphinx:
  configuration: docs/conf.py
python:
  version: 3.7
  install:
    - requirements: docs/requirements.txt

environment.yaml

channels:
  - conda-forge
dependencies:
  - python =3.7
  - pandoc

In summary

I have created beautiful documentation with readthedocs and sphinx, but I cannot get it to build on the web. How can I either: a) solve the issue of installing pandoc (is that the issue??) or b) remove the need for pandoc (e.g., is there a way to include jupyter notebook files in my docs without needing pandoc? How to do this?)?

Thanks so much for anyone's advice!

Answer 1

It's currently not very well explained on the PyPI page for pandocs, but pandoc is actually written in Haskell, not Python. The pandoc package for python which you can install with pip install pandoc doesn't come with an executable copy of pandoc, it is just a wrapper to provide bindings to pandoc within python. It does this by making shell calls to your system copy of pandoc, and so you need to separately install a copy of the actual pandoc executable to your system in order for the python bindings to have something to bind to!

For example, on Debian/Ubuntu, you will need to install a system copy of pandoc using apt in addition to installing the python bindings with pip.

sudo apt install pandoc
pip install pandoc

Though you can do this on your local system, this isn't a viable solution for getting it to work on readthedocs.

---

In contrast to this, conda tries to make packages work irrespective of things installed on the rest of your system. Hence the conda pandoc package ships with an actual copy of pandoc, which it uses instead of trying to use your system copy of pandoc. If you're using conda, there is only one command you need to run to get pandoc working within python:

conda install pandoc

If you're using a conda environment for your readthedocs specification (as is the situation for OP), you can just add pandoc to the conda section instead of the pip section of the env.

---

The pandoc package on PyPI which you can install with pip isn't actually an official pandoc package for python. There's also pypandoc and py-pandoc, which are equally unofficial.

Supposedly py-pandoc is a one-stop shop for installing pandoc through a python interface. I think you just have to do pip install py-pandoc or conda install pandoc and it works immediately, though I'm not sure due to the lack of documentation.

For myself, I'm using Sphinx to build documentation. My personal solution to the problem of installing pandoc when building python documentation (including on readthedocs) is to add the following code block to conf.py.

from inspect import getsourcefile

# Get path to directory containing this file, conf.py.
DOCS_DIRECTORY = os.path.dirname(os.path.abspath(getsourcefile(lambda: 0)))

def ensure_pandoc_installed(_):
    import pypandoc

    # Download pandoc if necessary. If pandoc is already installed and on
    # the PATH, the installed version will be used. Otherwise, we will
    # download a copy of pandoc into docs/bin/ and add that to our PATH.
    pandoc_dir = os.path.join(DOCS_DIRECTORY, "bin")
    # Add dir containing pandoc binary to the PATH environment variable
    if pandoc_dir not in os.environ["PATH"].split(os.pathsep):
        os.environ["PATH"] += os.pathsep + pandoc_dir
    pypandoc.ensure_pandoc_installed(
        quiet=True,
        targetfolder=pandoc_dir,
        delete_installer=True,
    )


def setup(app):
    app.connect("builder-inited", ensure_pandoc_installed)

with pypandoc appearing in the requirements.txt file

This tells Sphinx to check to see if pandoc is already on the system path, and if it isn't available then it will download and install a copy to the docs/bin/pandoc directory of the repository and add it to PATH.

The advantage of this method is it should be universal, enabling any downstream user to build the documentation. Whilst for your readthedocs build you can control that pandoc was installed with conda, you can't guarantee that every user who wants to build the documentation has installed the pandoc python package this way, nor that they have the Haskell pandoc package installed on their system.

Answer 2

I faced the same issue.

Notebook error:
PandocMissing in getting_started.ipynb:
Pandoc wasn't found.
Please check that pandoc is installed:
https://pandoc.org/installing.html

It seems there is a difference whether you install pandoc via pip or if you use conda.

Try the following:

pip uninstall pandoc
conda install pandoc

Using the conda version solved the issue on my machine.

On a different front, if you are using poetry, you may also come across the same error message, i.e. PandocMissing.

Windows:
Install Pandoc via its Windows installer: https://pandoc.org/installing.html.

Linux (tested with Ubuntu 2204):
sudo apt install pandoc

Answer 3

first

pip uninstall pandoc

next,

conda install pandoc

just do it in Anaconda prompt, (restart kernel if necessary), and done!

Answer 4

This can be done by the non-Conda, pip fanboys.

I finally hit the wall and relented to allow Miniconda to play a small role in managing packages for my python3.1 on a Windows PC. I could not get Jupyter-book to convert a markdown file to a mySt markdown file. And the issue was with pandoc.

But I will always remain determined to continue controlling my python's packages myself using pip. I will not give control to Anaconda, and I will continue to control python's packages myself.

With that long qualifier out of the way, I will explain what I did.

I uninstalled pandocs using pip:

pip uninstall pandocs

And then I installed miniConda: https://docs.conda.io/en/latest/miniconda.html There was a Windows Installer for my version of python, python3.10. It downloads into the Windows download folder.

Years ago, I fought Anaconda for control over my python and will not allow MiniConda any similar control. During the installation of Miniconda, I unselected two default checkboxes which would have given Miniconda control over vsCode, etc... And also did not accept the default installation destination and instead installed Miniconda to a new folder:

C:\miniConda 

Then needed to add a path to Miniconda by adding a new system variable. That was done by Editing System Properties - Environmental Variables.

C:\MiniConda\Scripts\conda.exe

Close the cmd prompt and then open again. At the folder holding the .md file, open the cmd prompt. The smallish sized cmd prompt is preferred over the large Terminal window. Run conda and install pandocs.

conda install pandoc

Finally, convert the .md to myst.

jupyter-book myst init myBook.md --kernel ir

It works. Miniconda is kept under control and the pandoc package worked. And the process created a mySt markdown file.

And for those who don't read instructions carefully, they will miss this mention that I'm using the r kernel rather than the python kernel.

Answer 5

Well, I got it working on my system:

• download and install pandoc from https://pandoc.org/installing.html
• start a new session (cmd or power shell)
• call sphinx make again

Simple as that. Hope you can get it working!