Running multiple Python versions (2.x. 3.y, 3.z) on Jupyter Notebook via multiple kernels

Question

Apologies first-hand if I've done some silly mistake in the below raised issue. I have been stuck on this since quite some time, trying to successfully install multiple Python versions (via separate virtual environments) and run Jupyter notebook with all three versions in Change kernel switch.

AIM:

Setup Anaconda with Python 3.5.6 as default and create two virtual environments with Python 2.7.13 and Python 3.7.3 version and to be able to switch between these three Python versions on a Jupyter Notebook on Windows.

Process Followed: What I did (and ended in mess):

I first successfully installed Anaconda3 with Python 3.5.6 as default (installed in C:\ProgramData\Anaconda3) and set the PATH variables. Jupyter Notebook was up and running with an ipython kernel running from 'base' (or root) from

(base)  jupyter kernelspec list
Available kernels:
python_3.5.6          C:\Users\username\AppData\Roaming\jupyter\kernels\python_3.5.6

and kernel.json file was also mapped to the correct python version.

.

Then I created my first virtual environment (Python_2.7.13_ENV):

(base)  conda create --p C:\ProgramData\Anaconda3\envs\Python_2.7.13_ENV python=2.7.13

and installed jupyter on it

(base)  activate Python_2.7.13_ENV
(Python_2.7.13_ENV)  conda install notebook ipykernel
(Python_2.7.13_ENV)  python -m ipykernel install --p C:\Users\username\AppData\Roaming\jupyter\kernels\ --name Python_2.7.13_ENV --display-name "python_2.7.13"

I used the prefix notation as the default installation syntax was installing it for the root user and I wanted it to install it only for a specific user. And this worked like a charm. The updated jupyter kernelspec read:

(base)  jupyter kernelspec list
Available kernels:
python_3.5.6        C:\Users\username\AppData\Roaming\jupyter\kernels\python_3.5.6
python_2.7.13       C:\Users\username\AppData\Roaming\jupyter\kernels\python_2.7.13

and kernel.json file was also mapped to the correct python version ("C:\\ProgramData\\Anaconda3\\envs\\Python_2.7.13\\python.exe")

This was also working fine. I could open a file in jupyter and succesfully switch between the two kernels.

.

Than I followed the same steps for creating my second virtual environment (Python_3.7.3_ENV):

Now, the updated kernelspec read:

(base)  jupyter kernelspec list
Available kernels:
python_3.5.6     C:\Users\username\AppData\Roaming\jupyter\kernels\python_3.5.6
python_2.7.13    C:\Users\username\AppData\Roaming\jupyter\kernels\python_2.7.13
python_3.7.3     C:\Users\username\AppData\Roaming\jupyter\kernels\python_3.7.3

and the kernel.json was also mapped to the correct python version.

Problem:

• Both the virtual envs were created successfully.
• Now when I run a jupyter notebook and try to switch to Python 2.7.13 kernel, it works fine, but shows a ImportError: DLL load failed (due to some import issue in zmq) on switching to Python_3.7.3 kernel.
• However, when I first activate the Python_3.7.3_ENV virtual env and then load the jupyter notebook, I am able to switch between all three Python versions.

Can anybody provide a solution on how to toggle between all three versions without activating the virtual env beforehand if it's possible as I am able to do it with Py 2.7 & Py 3.5 versions.

PS. I have set the 'open with' default on right-click on a ipynb file to jupyter-notebook.exe.

Answer 1

I have similar setting: default python 3.7, env: python 3.6 and python 2.7.

• set multiple versions of python/ipykernels running on jupyter notebook

Check you Anaconda version, if it is '>= 4.1.0', it is easier since after 4.1.0, anaconda includes a special package nb_conda_kernels that detects conda environments with notebook kernels and automatically registers them. If the Anaconda version is lower than 4.1.0 or just want to manually do that, you can reference here.

• For python2.7 and adding the ipykernel to jupyter notebook, shown as 'Python (py27)'

conda create -n py27 python=2.7 ipykernel
conda activate py27
python -m ipykernel install --user --name=py27 --display-name "Python (py27)"

• For python3.6 and adding the ipykernel to jupyter notebook, shown as 'Python (py36)'

conda create -n py36 python=3.6 ipykernel
conda activate py36
python -m ipykernel install --user --name=py36 --display-name "Python (py36)"

• Then we can check if they works well not. You can deactivate from specific env and go back to the base env, and type jupyter notebook (or use graphic shortcut for jupyter notebook anaconda provides)and then you can create 'new' notebook, you can find apart from default Python 3, there also shows Python (py27) and Python (py36).

I was making mistake at the beginning to check which python was running:

Do not just use !python --version in jupyter notebook no matter which version kernel you are using.Because this is just like you running command in base env, it will always show the default env python version, which in my case is python 3.7.x.

What you can do to:

from platform import python_version
print(python_version())

## or this one, which ever you like. 
import sys
print(sys.executable)
print(sys.version)
print(sys.version_info)

You will get right python version accordingly.

• Show list of jupyter notebook kernels

jupyter kernelspec list (in base env, it will show all kernels name)

If you want to remove specific kernel, you can use:

jupyter kernelspec uninstall <kernel name>

• Another way to confirm the python version is right, you can check the ipykernel json file, if the path to start the python is right or not.

To do this: use jupyter kernelspec list, so you can get to know the path of json file. (for example, in my case I can get the path for py27: C:\Users\username\AppData\Roaming\jupyter\kernels\py27). Then, you can go to the dir/path, and you can check what is shown in kernel.json

for the argv, it should show the path to access related python version. For example,

• for py27, it will show like "C:\\Users\\username\\Anaconda3\\envs\\py27\\python.exe",

• for py36, it will show like "C:\\Users\\username\\Anaconda3\\envs\\py36\\python.exe"

• Last thing about setting PATH env variables: do not add Anaconda to the Windows PATH because this can interfere with other software. Instead, open Anaconda with the Start Menu and select Anaconda Prompt, or use Anaconda Navigator. More information can check the Anaconda official docs.

Hope this may be helpful if you meet similar issues.

PS: I was always using Unix like system, not so used to Windows env setting. That is what I met for setting different kernels for different anaconda envs and summarized it. Hope it helps.

Answer 2

After here and there, the only to make this work is having that virtual environment activated.

Following the above steps, I have installed Python 2.7.13, 3.5.6 and 3.7.3 version. My default python is Python 3.5.6 while the other two versions are installed in two virtual environments - Python_2.7.13_ENV and Python_3.7.3_ENV respectively.

How to use them ?

• For Python 2.7 and 3.5, just use the Jupyter notebooks normally as you would. Since the default python is set to Python 3.5.6 there is no problem in switching between two versions using Change Kernel option in Jupyter Notebook Toolbar.

• For Python 3.7 we first need to activate Python_3.7.3_ENV virtual environemnt and then we can switch between all three versions succesfully using Change Kernel option in Jupyter Notebook.