16

I isolate my data science projects into virtual environments using pipenv. However, running a Jupyter notebok does not access the local environment and uses the default IPyKernel. I've seen that you can register virtual environments from within the environment, but this requires installing the ipykernel package which itself requires Jupyter!

Is there anyway to avoid this and just use a single Jupyter install for all virtual environments?

user126350
  • 505
  • 5
  • 11
  • No.. You have to install separate jupyter notebook for each virtualenv. As a virtualenv is activated, the jupyter notebook installed for that particular virtualenv will be available in path. – SaiNageswar S Sep 29 '18 at 08:00
  • Any idea why this is the case? Seems like a waste of disk space and requires configuring add-ons etc for each new project... – user126350 Sep 29 '18 at 08:01
  • pip install puts all files in site_packages of virtualenv. All dependencies available in particular virtualenv will only be available in that particular jupyter notebook instance. – SaiNageswar S Sep 29 '18 at 10:04
  • That gave me an idea to just add the virtual environment's `site_packages` to the Python path. Is this going to be regrettable? – user126350 Jan 29 '19 at 00:14

3 Answers3

2

Generally, you'd install jupyter once and do the following in your virtual environments: 

pip install ipykernel  
python -m ipykernel install --user

This isn't enough when you're running multiple Python versions.
There's a guide here that tries to address this:
https://medium.com/@henriquebastos/the-definitive-guide-to-setup-my-python-workspace-628d68552e14

It's not 100% failsafe, but it can help you avoid reinstalling jupyter notebook all the time.

Hendrik D
  • 71
  • 7
1

I found that there are few problems when reinstall jupyter for each environment separately: i.e. pip install jupyter jupyterlab in new environments.

I had multiple issues (with and without Conda), where Jupyter would install packages to a different python environment when you use !pip install a_package_name within a cell. The shell environment still kept track of the non-environment python, and you can tell this by comparing the outputs of !which python and

import sys
sys.executable

Therefore, when you tried to import the package, it would not be available, because the cells used the environment python/ kernel (as it detected the venv directory).

Ben Butterworth
  • 22,056
  • 10
  • 114
  • 167
0

I found a workaround that I'd appreciate feedback on. I changed pipenv to install virtual environments into the working directory by add to .bashrc/.bash_profile:

export PIPENV_VENV_IN_PROJECT=1

Now when opening a Jupyter notebook, I simply tack on the virtual environment's packages to the Python path:

import sys
sys.path.append('./.venv/lib/python3.7/site-packages/')

Is this a terrible idea?

user126350
  • 505
  • 5
  • 11