1

I'm currently using Jupyter notebooks with Python and pipenv virtual environments. I'm using the following code as a solution for loading the virtual environment inside my Jupyter notebook:

import sys
sys.path = ['./.venv/lib/python37.zip',
            './.venv/lib/python3.7',
            './.venv/lib/python3.7/lib-dynload',
            './.venv/lib/python3.7/site-packages',
] + sys.path

Is this bad practice? If so, what are the problems, risk, side-effects etc?

(Note this works because I've configured pipenv to store the environment in my working directory via echo "export PIPENV_VENV_IN_PROJECT=1" >> ~/.bash_profile)

How did I get here?

My goal is to use Jupyter notebooks for numerous Python (and R) projects, each project with its own virtual environment. I've encountered two unsatisfactory solutions for achieving this goal:

  1. Install Jupyter in every virtual environment.

    • Slow installation process
    • Requires configuration of extensions every time
    • Wastes disk space as multiple versions are installed over time
    • Fills virtual environment with irrelevant packages

  2. Create an IPyKernel in every virtual environment.

    • Requires installing IPyKernel in the virtual environment, which installs most of Jupyter anyway
    • Again, filling the environment with irrelevant packages
    • Requires an additional command to register the new kernel with Jupyter
    • Requires the user to name each kernel manually
    • Requires the user to select the appropriate Kernel each time they open a notebook

My third solution is to change the above configuration for pipenv, so the virtual environment is located in the working directory. This way, my generic kernel can use the same paths to target the appropriate virtual environment. I prepend the appropriate paths to the sys.path in Jupyter. Is this a terrible idea?

user126350
  • 505
  • 5
  • 11
  • Considering I've tried the above solutions myself and I, too, feel unsatisfied with the wasted space and long time for installations, I think your approach is worth something to me. That being said, I think one possible mess-up you create is **accidentally importing or loading a Python library from your base (conda?) environment** instead of venv in the event which a library is installed in base but not venv. I'm not experienced enough to warrant a thorough test, though I suspect it might happen. – Nuclear03020704 Dec 17 '21 at 06:12

1 Answers1

0

It could definitely mess someone else up if you're sharing the notebook with them and they don't have their filesystem laid out the exact way you do. Could you just create a new virtual environment and then symlink to the one that has jupyter installed?

Matt Whitehead
  • 88
  • 1
  • 1
  • 6
  • I expect it wouldn't adversely effect others since invalid paths are ignored by sys.path, although it may confuse them. That said, I could also set this to run on load for just my Jupyter installation. I'm not sure I understand how the symlink solution would get the packages in the virtual environment running in Jupyter? – user126350 Jan 29 '19 at 16:20