0

I have deployed jupyterhub to GKE using the Zero to Jupyterhub Helm chart.

I've set up my python environment and notebooks in my spawned singleuser instance, and now I would like to remotely (e.g., via API on another server) run a notebook in my singleuser environment and then download the outputs.

If it helps, I've parameterized the notebook to run it with papermill because I'd like to write a script to run the notebook over series of datasets.

RAY
  • 474
  • 4
  • 21
  • https://stackoverflow.com/a/55504503/260826 – gogasca Jan 27 '21 at 08:43
  • Thanks for the link. My issue is not running papermill, but kicking off a run from a separate server. I'm hoping to find an API (or equivalent) I can use to kick off the jupyter/papermill command. – RAY Jan 28 '21 at 15:02

1 Answers1

0

Have a look here. This allows you to deploy your Jupyter notebook as serverless function. You can then invoke that serverless function with different parameters (datasets) on demand.

Prakash Gupta
  • 94
  • 6
  • Thanks. That looks similar to what I'm currently doing in that I'm downloading the notebook from its original source and running it in a different environment. I want to run it in its original environment so that I have access to other source code that gets imported into the notebook. This additional source would not be pip installed packages. – RAY Mar 09 '21 at 06:31
  • Got it. How complex is the file structure for this "additional source" ? If it's just a few files in same directory as the notebook, I believe that can be bundled in the serverless function along with the notebook itself. – Prakash Gupta Mar 11 '21 at 03:59
  • It's arbitrary. I'm providing a jupyter environment to 3rd parties and I want to run something they build in their environments from a central system. – RAY Mar 12 '21 at 04:55
  • If along with Jupyter Notebook and requirements.txt, you also get option to specify some support files (any kind of files). These support files then become part of the environment (along with Jupyter Notebook and dependencies) in which the serverless function executes. Will that address your use case? – Prakash Gupta Mar 12 '21 at 10:37
  • It would, but at that point, I might as well zip up the whole environment and download it (or mount the disk it runs on, which is something I explored as well). For security/performance reasons, it would be preferable to run the notebook in its original environment. – RAY Mar 17 '21 at 19:38