Stack Overflow
Stack Overflow

Questions tagged [papermill]

Papermill is a tool for parameterizing and executing Jupyter Notebooks.

Help / Support / How do I?

For help and support or technical, please submit your questions to Stack Overflow: https://stackoverflow.com/questions/tagged/papermill.

Also check out the papermill FAQ before asking a question.

When looking for info, especially build info, one needs to check in the GitHub site.

Report Issues

Please report bugs, feature requests and installation / compatibility issues on the papermill issues tracker on GitHub. If you need help with using papermill, please do not use the issue tracker for that. Instead, direct your questions to Stack Overflow.

Documentation

Take a look at the Papermill documentation on ReadTheDocs.

72 questions
81
votes
11 answers

Passing command line arguments to argv in jupyter/ipython notebook

I'm wondering if it's possible to populate sys.argv (or some other structure) with command line arguments in a jupyter/ipython notebook, similar to how it's done through a python script. For instance, if I were to run a python script as…
justadampaul
  • 979
  • 1
  • 7
  • 11
41
votes
12 answers

How to run a Jupyter notebook with Python code automatically on a daily basis?

I have some Python code in a Jupyter notebook and I need to run it automatically every day, so I would like to know if there is a way to set this up. I really appreciate any advice on this.
Betsy Curbelo
  • 559
  • 1
  • 5
  • 10
10
votes
1 answer

How to avoid Keyerror named 'kernelspec' in Papermill?

I am running a papermill command from withing airflow(docker). The script is stored on S3 and I run it using Python client of papermill. It ends up in an error which is not at all understandable: Traceback (most recent call last): File…
Aviral Srivastava
  • 4,058
  • 8
  • 29
  • 81
8
votes
2 answers

Hung cells: running multiple jupyter notebooks in parallel with papermill

I am trying to run jupyter notebooks in parallel by starting them from another notebook. I'm using papermill to save the output from the notebooks. In my scheduler.ipynb I’m using multiprocessing which is what some people have had success with. I…
sean5446
  • 133
  • 2
  • 6
8
votes
1 answer

how to pass parameters to PapermillOperator to run job on airflow?

While running airflow job using PapermillOperator dag execution getting failed. I am having problem in passing parameters to PapermillOperator. I opened papermill_operator.py (packages/airflow/operators/papermill_operator.py) and hardcoded one line…
Umesh Gaikwad
  • 301
  • 3
  • 14
6
votes
1 answer

ERROR: jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3 occurs in Airflow/papermill running as a docker container

I am trying to run an example papermill notebook that you can find here: at the Official Apache Airflow url airflow-github with a corresponding dag file, found within the same directory as specified above. I am running this locally using a…
Bruce Macdonald
  • 61
  • 1
  • 3
6
votes
2 answers

How to specify kernel while executing a Jupyter notebook using Papermill's Python client?

I am running a jupyter notebook script using Papermill's python client. While running, I want to specify the kernel. However, the script ran within the same kernel as my active python shell(through which I made the API call to papermill in order to…
Aviral Srivastava
  • 4,058
  • 8
  • 29
  • 81
5
votes
1 answer

Papermill PySpark support

I'm looking for a way to easily execute parametrized run of Jupyter Notebooks, and I've found Papermill Project (https://github.com/nteract/papermill/) This tool seems to match my requirements, but I can't find any reference for PySpark kernel…
Mattia Graziani
  • 51
  • 3
4
votes
2 answers

Adding tags to Jupyter notebooks

As suggested in the description of the Jupyter Miscrosoft extension for Visual Studio Code (VSC), I come here to ask you about support for cell metadata in jupyter notebooks. It seems nothing is currently available for adding cell metadata and,…
Antonio Sesto
  • 2,868
  • 5
  • 33
  • 51
4
votes
1 answer

zmq.error.ZMQError: Address already in use, when running multiprocessing with multiple notebooks using papermill

I am using the papermill library to run multiple notebooks using multiprocessing simultaneously. This is occurring on Python 3.6.6, Red Hat 4.8.2-15 within a Docker container. However when I run the python script, about 5% of my notebooks do not…
Dan
  • 63
  • 1
  • 6
4
votes
0 answers

Kernel dies while executing papermill

I am trying to automatically generate several notebooks with papermill using the following code: template = "preprocessing_template.ipynb" parameters = {"allowed_values":[0,1], "epsilon":0.01, "to_csv":True} kernel_name =…
NikPapadopoulos
  • 41
  • 3
4
votes
1 answer

Connect to remote python kernel from python code

I have been using PaperMill for executing my python notebook periodically. To execute compute intensive notebook, I need to connect to remote kernel running in my EMR cluster. In case of Jupyter notebook I can do that by starting jupyter server with…
cyn0
  • 522
  • 2
  • 7
  • 23
4
votes
4 answers

Can the notebook generated from papermill be outputted with a live running kernel?

When papermill generates a notebook, a .ipynb file is created in the output path that says it is not running in the jupyter home page. I would prefer that when the notebook has finished executing, it remains running with a live kernel so I can…
rmalhotra
  • 41
  • 3
4
votes
2 answers

Is it possible to halt execution of a Jupyter notebook using papermill?

I have a set-up whereby I have a few Jupyter notebooks that are parameterised so I can run them from another notebook. I'm using the papermill module to do this which is quite convenient. The simplest way of running looks something like this path =…
mcansado
  • 2,026
  • 4
  • 25
  • 39
4
votes
1 answer

Pass arguments to jupyter notebook started with nbformat

I am a beginner in Jupyter. I have a Python program that starts up my notebook using the nbformat execute API. It works great, but there is one thing I have not managed to figure out; how can I pass data from my program to the notebook I am about to…
Mr. Developerdude
  • 9,118
  • 10
  • 57
  • 95
1
2 3 4 5