16

I am working in notebooks provided in the Workbench section of Vertex AI. I need an updated version of Python, but I only have access to Python 3.7 in these notebooks. I have successfully followed these steps and if I run python3.8 --version in terminal, I get Python 3.8.2, which is good, but python --version still returns Python 3.7.12. If, following this answer and restarting notebook's kernel, I run

from platform import python_version
print(python_version())

in a notebook, and I get 3.7.12.

How do I get a notebook in Vertex AI supporting an up-to-date Python version?

zabop
  • 6,750
  • 3
  • 39
  • 84

4 Answers4

12
#create a new conda env:
$ conda create -n python38 python=3.8

#Activate your new Python 3.8 environment:
$ conda activate python38

#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel

Refresh the page and the new python38 env will be avaiable: enter image description here enter image description here

ewertonvsilva
  • 1,795
  • 1
  • 5
  • 15
  • This answer worked when I accepted it, but no longer does when I tried again with 3.10. – zabop Sep 08 '22 at 19:55
  • or with 3.8, for that matter. – zabop Sep 09 '22 at 06:35
  • This answer works correctly for me. However if I stop and restart my workbench I loose the conda environment created. If I look for the kernel in the top right I can still see python38 but it returns me the error: failed to load kernel. By listing all conda environments in conda, however, python38 is not yet available. Do you have any idea on how to save the environment to use it after a workbench restart? – Marco Abbatangelo Dec 16 '22 at 09:50
  • 2
    Added an issue; please +1 it: https://issuetracker.google.com/issues/249804661 – Brian Bien Dec 17 '22 at 15:40
10

For me, @ewertonvsilva's answer wasn't working. I had to install the ipython kernel at user level then deactivate the kernel before refreshing the page to make the environment appearing on Jupyter Lab, All commands combined :

# create a new conda env:
$ conda create -n python38 python=3.8

# Activate your new Python 3.8 environment:
$ conda activate python38

#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel

# install the ipython kernel at user level    
(python38)$ ipython kernel install --user --name=python38

# Deactivate the new environment
(python38)$ conda deactivate

Then refresh the page.

Sources :

How to add conda environment to jupyter lab

Installing the IPython kernel - IPython Documentation

Innat
  • 16,113
  • 6
  • 53
  • 101
Gilles Charlier
  • 101
  • 1
  • 5
8

After a few trials and errors, I found that:

  • Changes under /opt/conda will not be preserved when you change/stop instances.
  • Custom conda env needs to be saved to $HOME.
  • conda activate $ENV will not work, since conda init was not executed.
  • pip install installs to /opt/conda/lib/python3.7/site-packages, which will not be preserved.
  • pip install --user installs to ~/.local/lib/python3.7, which will be applied to every python3.7 kernel, kinda messy.
  • In notebook, !pip uses pip in shell, not in current kernel.
  • In notebook, %pip uses pip in current kernel.
  • In custom env, don't use --user flag, or it installs to ~/.local/lib.

Run the commands in system terminal:

VENV=new_env

# create new env in `$HOME`
conda create -y -q -p $HOME/conda_env/$VENV python=3.8 ipykernel

# activate env
source /opt/conda/bin/activate ~/conda_env/$VENV

# register kernel to `$HOME/.local/share/jupyter/kernels`, so it will be preserved
python -m ipykernel install --user --name $VENV

# install your packages, WITHOUT `--user`
pip install numpy==1.22

# check package installation path
pip list -v

Now you can change kernel in Launcher (takes a few minutes to refresh), or the box on the top right of notebook. The kernel will be preserved.

In notebook with custom env:

# use `%pip` in notebook, instead of `!pip`
%pip install numpy==1.22  # `/home/jupyter/conda_env/$VENV/lib/python3.8/site-packages`
!pip install numpy==1.22  # `/opt/conda/lib/python3.7/site-packages`
!pip install --user numpy==1.22  # `~/.local/lib/python3.7/site-packages`

# you may see the difference in custom env
%pip list -v
!pip list -v
Joseph Lee
  • 116
  • 1
  • 4
0

Python 3.10 is now installed by default on the latest image: https://cloud.google.com/deep-learning-vm/docs/release-notes

Brian Bien
  • 723
  • 8
  • 21