0

I have a Jupyter notebook that scrapes web data and saves a dataframe to a csv. I would like to run this every day automatically. I am using a mac for this project.

I have looked around a lot (including here: how to run a python jupyter notebook daily automatically), but as of yet I have not found a clear enough answer. I am quite new to all this, so I am looking for a step-by-step: like how you'd explain it to someone with no knowledge on cron etc.

Any advice would be much appreciated! Thank you!

Marina W.
  • 91
  • 2
  • 10

1 Answers1

2

Why don't you just convert the jupyter notebook into a raw python file? You can use this command: jupyter nbconvert --to script "[YOUR_NOTEBOOK].ipynb" (replace [YOUR_NOTEBOOK] with your notebook name)

EDIT: You could also use Juptext, as pointed out by @Wayne in the comments below.

If you need a jupyter notebook:

  1. Use datalab or papermill
  2. Use the SeekWall Chrome Extenstion
  3. Create a custom python script to launch jupyter notebook, and run that python script using the Automator app in your Mac
Ayush Garg
  • 2,234
  • 2
  • 12
  • 28
  • [Jupytext](https://github.com/mwouts/jupytext#command-line-conversion) on command line will allow you to convert to a script and back again, too. It can even execute notebooks, without requiring conversion. – Wayne Mar 07 '20 at 19:14
  • Nice. I'll add that to my answer. – Ayush Garg Mar 07 '20 at 19:15