2

Since Python version changes are far and few between, I always forget how I have created a new Conda environment with the latest Python for Jupyter Notebook, so I thought I'd list it down for next time. From StackOverflow, there are some answers that no longer worked, and below is a compilation of commands I found on StackOverflow that worked for me, Nov-29-2022. These instructions below are for Windows, and using Powershell (although they can also be used for the normal command line cmd.exe)

    # make sure you are in the base env

    # update conda
    conda update conda

    # to allow support for powershell
    conda init --all

    # The conda-forge repository seems to have at least the latest
    # stable Python version, so we will get Python from there.
    # add conda-forge to channels of conda.
    conda config --add channels conda-forge

    conda update jupyter
    # to fix 500 internal server error when trying to open a notebook later
    pip3 install --upgrade --user nbconvert

    # nb_conda_kernels enables a Jupyter Notebook or JupyterLab
    # application in one conda environment to access kernels for Python,
    # R, and other languages found in other environments.

    conda install nb_conda_kernels

    # I will now create a new conda env for Python 3.11 and name it as Python3.11
    conda create -n python3.11 python=3.11

    # check that it was created
    conda info --envs

    conda activate python3.11

    # Once installed, need to install ipykernel so Jupyter notebook can
    # see the new environment python3.11. 
    conda install -n python3.11 ipykernel

    # install ipywidgets as well for some useful functionalities 
    conda install -n python3.11 ipywidgets

    # Since I use R too, I'll also add a note here on R
    # To utilize an R environment, it must have the r-irkernel package; e.g.
    # conda install -n r_env r-irkernel

    # example to install a package in the new env, if desired
    # conda install --update-all --name python3.11 numpy

    #conda list will show the env's packages, versions, and where they came from too
    conda activate python3.11
    conda list
    conda deactivate

    # Now to check if the new environment can be selected in Jupyter
    # Notebook.  I change to the root directory first so jupyter 
    # notebook can see every folder.  Note that we are in base
    # environment, although no problem if in another environment 
    cd\
    jupyter notebook

    # If I open an existing notebook for example, I can tap on Kernel,
    # then Change kernel, and I should now be able to select the kernel
    # from the new environment I created, shown as "Python [conda env:python3.11]".
    #
    # There will also be another entry showing just the name of the env,
    # in this case, python3.11.  Just ignore this, select the entries
    # starting with "Python [conda env" ...   
    # 
    # If I tapped on New instead when Jupyter Notebook opened, it will
    # also show the list of envs. 

    # to check version, either use :
    !python --version

    # or

    from platform import python_version
    print(python_version()) 

    # both will show the Python version of whatever kernel is in use
    # by Jupyter notebook

    # to test Python 3.10 or 3.11 for example... from 3.10, an optional
    # strict parameter for zip has been added and can be used to
    # generate an error if lists' lengths are not the same

    a = [1,2,3,4]
    b = ['a', 'b', 'c']
    for val1, val2 in zip(a,b, strict = True):
        print(val1, val2)
    
    # this should appear - ValueError: zip() argument 2 is shorter than argument 1

Is there another way ?

J R
  • 436
  • 3
  • 7

1 Answers1

0

  1. The steps in the main question above is the nb_conda_kernels way. With nb_conda_kernels installed in the base environment, any notebook running from the base environment will automatically show the kernel from any other environment which has ipykernel installed. We only need one jupyter notebook, ideally installed in the base env.

  2. Not ideal way: The "quick and dirty method" is to install jupyter notebook in each env. "If you install jupyter in any environment and run jupyter notebook from that environment the notebook will use the kernel from the active environment. The kernel will show with the default name Python 3 but we can verify this works by doing the following."

import os

print (os.environ['CONDA_DEFAULT_ENV'])

  1. The "usual or easy way" is "to individually register each environment you want to show in your kernels list." There is no need to install nb_conda_kernels.

After creating the new env, python3.11 for example

`

conda activate python3.11  # make it active

conda install ipykernel    # needed for each env

conda install ipywidgets   # for additional jupyter functionalities

python -m ipykernel install --user --name python3.11 --display-name "Python 3.11 env"            # will install kernelspec python3.11

`

That's it, when running jupyter notebook, one will see "Python 3.11 env" as one of the env to select from.

NOTE:

The problem with this easy way is that running commands using ! like:

!python --version

!pip3 list

!conda list

will always refer to the env where jupyter notebook was started from, irregardless of whatever kernel version is currently selected (in use) by jupyter notebook.

So if we do this :

!pip3 install --upgrade numpy

numpy will be installed or upgraded in the env where jupyter notebook was started from. This is a problem if we are programatically trying to upgrade a package within jupyter notebook itself based on a condition, for example.

If instead, we used the nb_conda_kernels way, the above command will always install/upgrade in the env of the active kernel, irregardless of the env where jupyter notebook was started from.

So that's something to take note of, if one is installing packages in environments other than base.

Personally, I use both nb_conda_kernels way (nb_conda_kernels) and the usual/easy way. Just follow all the steps of the usual way, then the last step, before running jupyter notebook, is :

# make sure you are in base env
conda install nb_conda_kernels

So my kernel list in Jupyter notebook would look like :

Python 3 (ipykernel)

Python3.11 env

Python [conda env:python3.11]

etc.

I can select whichever kernel I want and it will work, while remembering the behavior I mentioned above.

If I want !pip3 install --upgrade to work on the packages in the base, while using a kernel whose version is different from the base (Python 3.8 for example), I would start notebook from the base env, and select kernel "Python3.11 env".

And if I want !pip3 install --upgrade to work on the packages in the env of a certain env, I can start notebook from any env, and select the kernel "Python [conda env:python3.11]".

J R
  • 436
  • 3
  • 7