(How) Can I run python 2.7 & 3.4 notebooks side-by-side in jupyter (ipython)?

Question

I would like to configure jupyter to allow me to run python 2.7 and 3.4 kernels at the same time side-by-side in two different notebooks (or maybe even switch from one to the other in a single notebook).

(1) Is this possible?

I am asking since it was suggested in Using both Python 2.x and Python 3.x in IPython Notebook that this would be possible, but no elaborate answer was provided on how exactly this would work.

You can also see in https://try.jupyter.org that it somehow appears to be very much feasible (and you can even switch from one python 2 kernel to a python 3 kernel). So I find it reasonable to assume that it is indeed possible (but please correct me if I am wrong).

(2) How is this done?

Previous answers (e.g. in Open IPython Notebook 2.7 and 3.4 in Parallel) recommend starting two different ipython notebook servers at two different ports. This, of course, is entirely logical and possible, but does NOT answer my question.

I managed to install ipython for python 2 & 3. I then got both kernels to show up in jupyter by calling the following in each respective python environment:

ipython kernelspec install-self

This created kernel.json files for me and I could now select either of them to create a new notebook in jupyter. Example of kernel.json for python 2:

{
 "display_name": "Python 2", 
 "language": "python", 
 "argv": [
  "/usr/local/opt/python/bin/python2.7", 
  "-m", 
  "ipykernel", 
  "-f", 
  "{connection_file}"
 ]
}

The problem is, when I start jupyter from a python 2 environment (with $PYTHONPATH defined), I can only run a python 2 kernel (the other one will crash when opening the corresponding notebook). The same, when I start jupyter from a python 3 environment (with $PYTHONPATH defined). This generally makes sense to me, but I would like to know how to start or configure jupyter to allow both kernels to be run from the same jupyter instance and still specify my own specific $PYTHONPATHs to be able to load my packages.

I figured that this is my actual problem - predefining $PYTHONPATH before I start jupyter. So I called:

unset PYTHONPATH
ipython notebook

I can now indeed run both kernels (py2&3) from a single jupyter instance, but I can't load python packages from specific local directories. I guess I could do this manually inside the notebook with:

import sys
sys.path.append("/some/path/lib/python2.7/site-packages")

but is this really the way I should be doing it? or can I configure additional directories I normally write into $PYTHONPATH somewhere else (e.g. that kernel.json file?

PS: Please don't ask why I need this - no, I don't really need it - but I still would like to know if and how this might be possible.

[EDIT]

Please note that I would not be comfortable installing some python management utility (such as Anaconda) to accomplish all of this, (unless it is the only way).

Answer 1

I have Python 2 and 3 installed in Anaconda where 3 is the main installation and 2 is an environment installation done after the main installation. By default when running 'ipython notebook' it runs only the version of python active in the shell in which it was launched. After running 'ipython kernelspec install-self' when Python2 was active and again when Python3 was active, kernels for both are available in IPython Notebook regardless of which version of Python was active in the shell from which it was launched.

Given that 'python2' is the name of my Python2 environment, 'activate env' is the anaconda command for activating an environment named 'env' and 'deactivate' is the command for deactivating the currently active environment (if any), the commands that I used to enable this configuration were:

activate python2
ipython kernelspec install-self
deactivate
ipython kernelspec install-self

sys.path in IPython Notebook changes dynamically and appropriately depending on the version of Python that it currently runs. When it runs Python2 the sys.path is:

['', 'C:\\Users\\tn\\Documents\\python', 'C:\\Anaconda3\\envs\\python2\\python27.zip', 'C:\\Anaconda3\\envs\\python2\\DLLs', 'C:\\Anaconda3\\envs\\python2\\lib', 'C:\\Anaconda3\\envs\\python2\\lib\\plat-win', 'C:\\Anaconda3\\envs\\python2\\lib\\lib-tk', 'C:\\Anaconda3\\envs\\python2', 'C:\\Users\\tn\\AppData\\Roaming\\Python\\Python27\\site-packages', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\Sphinx-1.3.1-py2.7.egg', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\cryptography-0.9.1-py2.7-win-amd64.egg', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\win32', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\win32\\lib', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\Pythonwin', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\setuptools-18.1-py2.7.egg', 'C:\\Anaconda3\\envs\\python2\\lib\\site-packages\\IPython\\extensions']

After toggling the kernel to Python3, sys.path becomes:

['', 'C:\\Users\\tn\\Documents\\python', 'C:\\Anaconda3\\python34.zip', 'C:\\Anaconda3\\DLLs', 'C:\\Anaconda3\\lib', 'C:\\Anaconda3', 'C:\\Anaconda3\\lib\\site-packages', 'C:\\Anaconda3\\lib\\site-packages\\Sphinx-1.3.1-py3.4.egg', 'C:\\Anaconda3\\lib\\site-packages\\cryptography-0.9.1-py3.4-win-amd64.egg', 'C:\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Anaconda3\\lib\\site-packages\\Pythonwin', 'C:\\Anaconda3\\lib\\site-packages\\setuptools-18.1-py3.4.egg', 'C:\\Anaconda3\\lib\\site-packages\\IPython\\extensions']

PYTHONPATH=C:\Users\tn\Documents\python is included in sys.path for both versions of Python.

In order to customize sys.path depending on Python version for IPython Notebook, the environment variable PYTHONSTARTUP (Python2) (also see PYTHONSTARTUP (Python3)) can be used. This variable should contain a readable Python script that will be executed during startup of interactive mode. For use with both Python 2 and 3 it must be executable by both. For example, in C:\Users\tn\Documents\python I created the following script named 'start' that determines the major version of Python executing it and appends a path ending in that to sys.path:

import sys
v = sys.version_info[0]
if v == 2 or v == 3:
   sys.path.append(r'C:\Users\tn\Documents\python' + '\\' + str(v))

Now, on restarting IPython Notebook, with Python 2 sys.path[-1] is C:\Users\tn\Documents\python\2 while for Python 3 it is C:\Users\tn\Documents\python\3.