18

I have been using "ipython --script" to automatically save a .py file for each ipython notebook so I can use it to import classes into other notebooks. But this recenty stopped working, and I get the following error message:

`--script` is deprecated. You can trigger nbconvert via pre- or post-save hooks:
ContentsManager.pre_save_hook
FileContentsManager.post_save_hook
A post-save hook has been registered that calls:
ipython nbconvert --to script [notebook]
which behaves similarly to `--script`.

As I understand this I need to set up a post-save hook, but I do not understand how to do this. Can someone explain?

Louic
  • 2,403
  • 3
  • 19
  • 34

3 Answers3

22

[UPDATED per comment by @mobius dumpling]

Find your config files:

Jupyter / ipython >= 4.0

jupyter --config-dir

ipython <4.0

ipython locate profile default

If you need a new config:

Jupyter / ipython >= 4.0

jupyter notebook --generate-config

ipython <4.0

ipython profile create

Within this directory, there will be a file called [jupyter | ipython]_notebook_config.py, put the following code from ipython's GitHub issues page in that file:

import os
from subprocess import check_call

c = get_config()

def post_save(model, os_path, contents_manager):
    """post-save hook for converting notebooks to .py scripts"""
    if model['type'] != 'notebook':
        return # only do this for notebooks
    d, fname = os.path.split(os_path)
    check_call(['ipython', 'nbconvert', '--to', 'script', fname], cwd=d)

c.FileContentsManager.post_save_hook = post_save

For Jupyter, replace ipython with jupyter in check_call.

Note that there's a corresponding 'pre-save' hook, and also that you can call any subprocess or run any arbitrary code there...if you want to do any thing fancy like checking some condition first, notifying API consumers, or adding a git commit for the saved script.

Cheers,

-t.

Community
  • 1
  • 1
Tristan Reid
  • 5,844
  • 2
  • 26
  • 31
  • This works great, thanks! I had to remove my old profile_default and create a new one to get it working. – Louic Apr 01 '15 at 11:43
  • Cool, glad it worked! For future reference, if you have a default profile that doesn't have the config files in it, you can still run create profile on top of the existing profile. – Tristan Reid Apr 01 '15 at 16:54
  • 3
    **Update:** This solution is broken in iPython version 4, because of "The Big Split" of Jupyter from iPython. To adjust this solution to version 4, use the command `jupyter notebook --generate-config` to create a config file. The command `jupyter --config-dir` finds out which directory contains the config files. And the code snippet given by @TristanReid should be added to the file named `jupyter_notebook_config.py`. The rest works as before. – greg Oct 20 '15 at 20:58
  • 1
    I had an issue with configparser (`AttributeError: ConfigParser instance has no attribute 'read_file'`), had to upgrade to the last version (`pip install --upgrade configparser`) – Renaud Aug 09 '16 at 08:29
  • To check your version use `conda list jupyter` and check the number next to `jupyter-core` – Boern Apr 19 '17 at 07:49
  • Or if you installed with pip, use `pip freeze` – Tristan Reid Apr 19 '17 at 14:29
  • Is there an effective way to use `nbconvert` `--output-dir='path/to/package'` without hardcoding the path. For example to read it from a cell in the notebook? Or is it recommended to save notebooks in the package folder? – alancalvitti Aug 06 '19 at 14:07
  • @alancalvitti this seems like a completely separate question? But in any case, yes, you could execute nbconvert from a cell by passing a variable path: `!nbconvert --output-dir='$mypath'` – Tristan Reid Aug 09 '19 at 15:28
  • When I try to open a new notebook after editing `jupyter_notebook_config.py` I get ```Unexpected error while saving file: notebooks/Untitled.ipynb HTTP 500: Internal Server Error (Unexpected error while running post hook save: Command '['jupyter', 'nbconvert', '--to', 'script', 'Untitled.md']' returned non-zero exit status 1.)``` – ruslaniv Mar 16 '22 at 09:04
2

Here is another approach that doesn't invoke a new thread (with check_call). Add the following to jupyter_notebook_config.py as in Tristan's answer:

import io
import os
from notebook.utils import to_api_path

_script_exporter = None

def script_post_save(model, os_path, contents_manager, **kwargs):
    """convert notebooks to Python script after save with nbconvert

    replaces `ipython notebook --script`
    """
    from nbconvert.exporters.script import ScriptExporter

    if model['type'] != 'notebook':
        return

    global _script_exporter
    if _script_exporter is None:
        _script_exporter = ScriptExporter(parent=contents_manager)
    log = contents_manager.log

    base, ext = os.path.splitext(os_path)
    py_fname = base + '.py'
    script, resources = _script_exporter.from_filename(os_path)
    script_fname = base + resources.get('output_extension', '.txt')
    log.info("Saving script /%s", to_api_path(script_fname, contents_manager.root_dir))
    with io.open(script_fname, 'w', encoding='utf-8') as f:
        f.write(script)

c.FileContentsManager.post_save_hook = script_post_save

Disclaimer: I'm pretty sure I got this from SO somwhere, but can't find it now. Putting it here so it's easier to find in future (:

drevicko
  • 14,382
  • 15
  • 75
  • 97
  • Is the var `py_fname` needed? And in any case, this answer is very very very similar to an entry in Jupyter docs: https://jupyter-notebook.readthedocs.io/en/latest/extending/savehooks.html – mbdevpl Sep 24 '17 at 06:11
1

I just encountered a problem where I didn't have rights to restart my Jupyter instance, and so the post-save hook I wanted couldn't be applied.

So, I extracted the key parts and could run this with python manual_post_save_hook.py:

from io import open
from re import sub
from os.path import splitext
from nbconvert.exporters.script import ScriptExporter

for nb_path in ['notebook1.ipynb', 'notebook2.ipynb']:
    base, ext = splitext(nb_path)
    script, resources = ScriptExporter().from_filename(nb_path)
    # mine happen to all be in Python so I needn't bother with the full flexibility
    script_fname = base + '.py'
    with open(script_fname, 'w', encoding='utf-8') as f:
        # remove 'In [ ]' commented lines peppered about
        f.write(sub(r'[\n]{2}# In\[[0-9 ]+\]:\s+[\n]{2}', '\n', script))

You can add your own bells and whistles as you would with the standard post save hook, and the config is the correct way to proceed; sharing this for others who might end up in a similar pinch where they can't get the config edits to go into action.

MichaelChirico
  • 33,841
  • 14
  • 113
  • 198