1

I've created a Python virtual environment, and activate it by doing:

joe@joe-mint $ source ./venvs/deep-learning/bin/activate

Which turns the prompt into:

(deep-learning) joe@joe-mint $

Now whenever I run a python package or try to install one, the system seems to ignore the fact that it's in a virtual environment and does things system-wide:

(deep-learning) joe@joe-mint $ which pip
/usr/local/bin/pip

The same happens when I try to install new packages that aren't on my system; it installs them to the system files (i.e. /usr/bin) instead of the virtual environment.

What's wrong with my virtual environment? How do I get it to ignore system files and do everything inside the environment?

I've looked at this question which says to use an explicit flag when creating the virtual environment to make it use the local environment packages, but I used python-3.5 -m venv to create the virtual environment, and this flag is removed in this version as it's now a default option.

I've also looked at this question and can confirm that the VIRTUAL_ENV variable is set correctly in the activate file of the virtual environment.

Community
  • 1
  • 1
Myridium
  • 789
  • 10
  • 20
  • What's the result of `echo $PATH` ? activate is supposed to place the venv's binaries at the front of PATH – jonatan Jan 20 '17 at 12:35
  • @jonatan - Turns out that because I installed the venv without pip, the `pip` command was using the system's one. Apparently this causes it to install packages to the system despite being in the virtual environment. I don't really understand it, but I managed to solve it in my answer. – Myridium Jan 20 '17 at 12:39
  • Possible duplicate of [virtualenv does not include pip](http://stackoverflow.com/questions/34503686/virtualenv-does-not-include-pip) – holdenweb Jan 20 '17 at 12:40
  • @holdenweb - I deliberately installed the venv without pip because of [a bug](http://askubuntu.com/questions/488529/pyvenv-3-4-error-returned-non-zero-exit-status-1) that prevented me from doing otherwise. – Myridium Jan 20 '17 at 12:42
  • And that's what's wrong with your virtual environment – holdenweb Jan 20 '17 at 12:45
  • @holdenweb - I was expecting the system's `pip` to install things into the venv since it had been activated. – Myridium Jan 20 '17 at 12:46
  • As you've discovered that doesn't work because virtualenvs rely on `$PATH` modifications to pick up the environment. Since you are using 3.4 (the bug is fixed in 3.5 and later) you might instead want to consider installing `virtualenv` into your system Pyrthon (assuming root permissions) and the nusing that rather than `pyvenv`. – holdenweb Jan 20 '17 at 13:00
  • @holdenweb - I'm using 3.5, not 3.4. Also, I didn't actually use `pyvenv`. I used `python-3.5 -m venv` as described in my question. – Myridium Jan 20 '17 at 13:04
  • @holdenweb - I might be wrong in assuming it's the same bug! Sorry if that's the case. I'm not sure why I ran into the error; that linked bug is just my best guess. – Myridium Jan 20 '17 at 13:06
  • In your answer you say you created the virtualenv without `pip` due to a long-standing bug, but that bug doesn't apply to 3.5, doe sit? – holdenweb Jan 20 '17 at 13:07
  • @holdenweb - Alright, I'm not sure what caused the error, but installing without `pip` as directed on that question solved the problem. Initially I ran `python3.5 -m venv ./deep-learning` which gave the error: `Error: Command '['...../deep-learning/bin/python3.5', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1`, just like the linked bug. Installing without `pip` using the `--without-pip` flag worked. – Myridium Jan 20 '17 at 13:10

1 Answers1

1

Here was the problem:

It seems that if you run pip on a venv without a local pip installation, then it will default to the system's pip outside the venv. Even if you've activated a virtual environment, this seems to want to install packages on the system rather than in the venv.

Here was the solution:

  • First, I had to install the virtual environment without pip due to a bug that has long remained unresolved.

  • Second, I installed pip in the virtual environment as per the instruction here. However, doing so required using some temporary folders that for some reason my user didn't have access to. So this failed, and the only way I could get it to work was to become root.

    1. sudo su
    2. activate ..../venvs/deep-learning/bin/activate to activate the virtual environment.
    3. curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python as per the answer linked above.

    Although which pip now indicated the correct pip (inside the venv) was being used, running pip would use the system one! Deactivating (deactivate) and reactivating the venv solved this.

    Now it took me a while to realise that having installed this as root caused me permission errors when trying to install more packages using pip inside the virtual environment.

  • chown <user>:<group> -R ..../venvs/deep-learning/*

And that was it. After these steps, I could activate the venv and run pip correctly. It would use the pip inside the venv, and install packages inside the venv.

Community
  • 1
  • 1
Myridium
  • 789
  • 10
  • 20
  • 1
    This worked for me, but I also had to remove the system python paths from my $PYTHONPATH and replace it with the venvs path: $export PYTHONPATH=..../venvs/deep-learning/lib/python3.8/site-packages – Dirk May 23 '22 at 15:55