1

I have created a Managed Notebook on VertexAI using Nvidia's Merlin NGC. Then I created a virtual environment "dl4coders" work where I installed libraries FastAI, etc. However, every time I shut down and restart the notebook, my virtual environments are gone. Any idea why is this happening?

In the picture below, you can see that dl4coders kernel is appearing in the list. enter image description here

However, the whole environment is deleted from the backend.

enter image description here

Question is why my virtual environment keeps getting deleted automatically?

exan
  • 3,236
  • 5
  • 24
  • 38
  • Hi, do you have steps to make an virtual environment? I also believe this is intended as part of the resource management of managed notebook. Is there a way to save virual environment files to a certain directory ? – Nestor Ceniza Jr Nov 04 '22 at 13:28

1 Answers1

0

Managed Notebooks are container based:

  • JupyterLab server is a container
  • Each kernel is a container

My guess is that your venv is created inside any of those containers and getting deleted upon restart since containers are ephemeral. Can you try creating the venv in /home/jupyter. Example:

virtualenv /home/jupyter/env

/home/jupyter is a mount which its contents are preserved upon restart.

gogasca
  • 9,283
  • 6
  • 80
  • 125
  • While the contents of a venv are preserved, I have found that the kernelspec for the environment vaporizes, so the environment doesn't end up in the Launcher or the drop-down shown by the OP. While it should be easy enough, I haven't been able to sort out how get a post-startup script to re-expose kernels of existing environments, even when placing the kernelspec files in a valid location on the data (not boot) disk. (thread [here](https://stackoverflow.com/questions/74921748/expose-kernel-for-existing-environment-on-vertex-ai-user-managed-notebook-startu)) – user6135514 Dec 29 '22 at 12:10