0

While there is an option to schedule notebook runs under 'managed notebook' of Google cloud (vertex AI) I'm unable to see any option for scheduling run in 'user-managed notebooks'. Managed notebooks has some constraints, unlike user-managed notebooks, which allow customized use of dependencies/containers.

Is there a way to that we run codes as notebook on schedule basis automatically in Google cloud Vertex AI?

Bhanu Chander
  • 390
  • 1
  • 6
  • 16

2 Answers2

1

You can either manually configure Jupyter Notebooks to run on a schedule, or to let Cloud Composer do the heavy lifting.

Regarding the manual route, you can manipulate Jupyter Notebook to run on a schedule, plugin for scheduling files for recurring execution, schedule recurring Python script (converted from Jupyter) on GCP, Cloud Scheduler to turn cronjob, using Cloud Function & Pub/Sub & Cloud Scheduler, see this Stack Overflow thread for “How to run a Python notebook daily automatically”.

While using Cloud Composer offers a less manual approach, and is more scalable if need be, you can refer to this Stack Overflow thread for more information. There is a “Jupyter Notebook Manifesto: Best practices that can improve the life of any developer using Jupyter notebooks” blog post by Google Cloud that explains the product in depth and can be found here.

Sakshi Gatyan
  • 1,903
  • 7
  • 13
  • Cloud scheduler seems promising but I'm wondering how the other solutions like Papermill for jupyter Notebook works for GCP, given that compute instance may not be active always. It would be good if we have a tool that can automate compute instance only for specific duration in a day while it execute the scripts as per the schedule. Let me know if you have come across any solution for this too. Thanks. – Bhanu Chander Feb 03 '23 at 12:16
  • This feature is coming up in Summer. (Notebook executor support just as Managed Notebooks) – gogasca Mar 19 '23 at 03:03
0

Fortunately, this package Jupyter Scheduler overcomes the problem of scheduling user-managed notebooks on JupyterLab/GCP.

Ref: https://blog.jupyter.org/introducing-jupyter-scheduler-f9e82676c388

pradeepvaranasi
  • 151
  • 13