80

I am deploying a Python package, and I would like to run a simple test to see if all cells in my notebook will run without errors. I would like to test this via commandline as I have issues running a notebook in virtualenv. Is there are simple command-line way to test this?

Note to the moderator: this question has been marked as a duplicate of How to run an .ipynb Jupyter Notebook from terminal? . However, this question was posted (asked Feb 18 '16 at 2:49) several days before that one (asked Feb 22 '16 at 3:35). At most, that post might be marked as a duplicate, and if deemed so, an appropriate action would be to merge the two questions, maintaining this, the original, as the master.

However, these questions may not be duplicates (the intent of the other author is unclear). Regardless, this question and it's answers specifically address executing cells within a jupyter notebook from the terminal, not simply converting notebooks to python files.

pylang
  • 40,867
  • 14
  • 129
  • 121
  • Related discussion: [Run Jupyter Notebook (.ipynb) from command line as if it were a .py file](https://stackoverflow.com/q/43602938/320399) – blong Aug 30 '17 at 03:04
  • Possible duplicate of [How to run an .ipynb Jupyter Notebook from terminal?](https://stackoverflow.com/questions/35545402/how-to-run-an-ipynb-jupyter-notebook-from-terminal) – Ciro Santilli OurBigBook.com Dec 12 '17 at 12:51
  • If using pytest, [`nbsmoke`](https://github.com/pyviz/nbsmoke) is a plugin that claims to accomplish this. – pylang Mar 12 '19 at 00:41

4 Answers4

132

nbconvert (a jupyter tool for notebook conversion) allows you to do this without any extra packages:

Just go to your terminal and type

$ jupyter nbconvert --to notebook --inplace --execute mynotebook.ipynb

Source

(Thanks Stephan for suggesting the --inplace flag)

NOTE: This said, I'd try to convert everything you need to a proper script. Jupyter notebooks are thought for exploring and sharing results, and not as a replacement of traditional programs.

finiteautomata
  • 3,753
  • 4
  • 31
  • 41
  • I like the idea of using nbconvert more than a separate package, although when I tried this I ran into [this issue](https://github.com/Anaconda-Platform/nb_conda_kernels/issues/34) – Max Power Feb 04 '17 at 16:15
  • 20
    You can use the --inplace flag as well: `jupyter nbconvert --to notebook --execute --inplace mynotebook.ipynb` – Stephan Schielke Jul 10 '17 at 12:25
  • 27
    Nbconvert will fail if any cell takes longer than 30s to run, you may want to add `--ExecutePreprocessor.timeout=600`. – bckygldstn Jan 24 '19 at 17:14
  • 4
    Documentation or this and programmatic ways of running notebooks can be found at https://nbconvert.readthedocs.io/en/latest/execute_api.html#module-nbconvert.preprocessors – TomDotTom Mar 05 '19 at 19:26
  • 1
    @bckygldstn Any way to drop the timeout all together? Or would I just need to set a really high value? – stefanbschneider Feb 01 '20 at 17:22
  • 1
    @CGFoX The timeout can be removed by setting to `None` or `-1` as per https://nbconvert.readthedocs.io/en/latest/execute_api.html#execution-arguments-traitlets – bckygldstn Feb 19 '20 at 21:27
  • 1
    Another helpful flag is `--ExecutePreprocessor.kernel_name=julia-1.6`, or whatever kernel you need. – M. Thompson May 23 '21 at 17:25
  • This doesn't seem to work for me. I have `print()` statements in my notebook and none of these are output when "executing" the notebook. If I "Download As" > ".py" and run that via python, then I see the print statements as expected. So..how to do the same thing from the command line? – sh37211 Sep 14 '21 at 19:44
  • @sh37211 I suggest you create a script from your notebook. This method is not flawless, and if you expect to have some output then a notebook is not a good fit for you – finiteautomata Sep 14 '21 at 22:39
44

You can use runipy to do this.

runipy will run all cells in a notebook. If an error occurs, the process will stop.

$ pip install runipy

$ runipy MyNotebook.ipynb

There are also commands for saving the output file as a notebook or an html report:

$ runipy MyNotebook.ipynb OutputNotebook.ipynb

$ runipy MyNotebook.ipynb --html report.html

Community
  • 1
  • 1
RajeshM
  • 872
  • 11
  • 21
17

You can also try papermill which allows you to execute notebooks from command line, and also pass parameters:

For example:

$ papermill mynotebook.ipynb mynotebook_output.ipynb -p start "2017-11-01" -p end "2017-11-30"

You can also run it without passing any parameter.

Giacomo
  • 1,796
  • 1
  • 24
  • 35
  • 2
    papermill has a nice console ui with execution status and times, so I actually recommend using it instead of `nbconvert` if you want some executing monitoring – M.Winkens Aug 17 '20 at 09:58
  • Does not execute in the usual sense. Print statements in the notebook do not go to stdout during execution, like converting to .py & running would do. Instead what you get is a whole new notebook file. ?? – sh37211 Sep 14 '21 at 19:47
0

You could use nbconvert to convert the ipynb file into a python script and then run it.

jupyter nbconvert --to python notebook.ipynb

python3 notebook.py

Abhilash
  • 85
  • 3
  • 11
  • [Jupytext](https://jupytext.readthedocs.io/en/latest/) also converts notebooks to scripts and has some other abilities that make it a nice tool to have in your toolbox. It can execute notebooks from the command line directly as well. – Wayne Sep 21 '22 at 19:33