Why is pip, inside a virtualenv, writing to /usr/lib?

Question

I'm running Python on a slightly older OS, whose system Python is at version 2.6. Hence I have 2.7 installed in my home directory and used that Python to get pip, and used that pip to install virtualenvwrapper. So I have

$ which python pip virtualenv virtualenvwrapper.sh
/home/user/bin/python
/home/user/bin/pip
/home/user/.local/bin/virtualenv
/home/user/.local/bin/virtualenvwrapper.sh

I use these tools to create a virtualenv, and try to install a module

$ mkvirtualenv fred
New python executable in fred/bin/python2.7.10
Also creating executable in fred/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/user/.virtualenvs/fred/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/user/.virtualenvs/fred/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/user/.virtualenvs/fred/bin/preactivate
virtualenvwrapper.user_scripts creating /home/user/.virtualenvs/fred/bin/postactivate
virtualenvwrapper.user_scripts creating /home/user/.virtualenvs/fred/bin/get_env_details

$ workon fred
$ export PYTHONPATH=/home/user/.virtualenvs/fred/lib/python2.7/site-packages
$ /home/user/.virtualenvs/fred/bin/easy_install --prefix=/home/user/.virtualenvs/fred pip
Creating /home/user/.virtualenvs/fred/lib/python2.7/site-packages/site.py
Searching for pip
Best match: pip 7.1.2
Adding pip 7.1.2 to easy-install.pth file
Installing pip script to /home/user/.virtualenvs/fred/bin
Installing pip3.4 script to /home/user/.virtualenvs/fred/bin
Installing pip3 script to /home/user/.virtualenvs/fred/bin

Using /home/user/.virtualenvs/fred/lib/python2.7/site-packages                                                                                                           
Processing dependencies for pip
Finished processing dependencies for pip

$ which python pip 
/home/user/.virtualenvs/fred/bin/python
/home/user/.virtualenvs/fred/bin/pip

$ pip install itsdangerous
Collecting itsdangerous
Installing collected packages: itsdangerous

Exception:
Traceback (most recent call last):
  File "/home/user/.local/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/home/user/.local/lib/python2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/home/user/.local/lib/python2.7/site-packages/pip/req/req_set.py", line 646, in install
    **kwargs
  File "/home/user/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 803, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/home/user/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 998, in move_wheel_files
    isolated=self.isolated,
  File "/home/user/.local/lib/python2.7/site-packages/pip/wheel.py", line 341, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/home/user/.local/lib/python2.7/site-packages/pip/wheel.py", line 319, in clobber
    shutil.copyfile(srcfile, destfile)
  File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/itsdangerous.py'

Why is pip trying to install the module into /usr/lib? Shouldn't it be installed to ~/.virtualenvs/fred/lib, or ~/lib?

Status as of Jul 4th, 2018: Unreproducible, as I do not have access to that company's servers any more, and I have not seen it on other systems since.

Not necessarily a solution, but you might be able to get away with installing packages at a user level. `pip install --user itsdangerous` — ThorSummoner, Oct 14 '15 at 16:12
@ThorSummoner - can't use `--user` as I need it installed in the virtualenv, not ~/.local — jalanb, Oct 14 '15 at 16:17
Your prompt seems to indicate that the virtualenv is not activated. — Robert Moskal, Oct 14 '15 at 16:20
That's only the 2nd line of my prompt, and the `which` commands show that the virtualenv is activated — jalanb, Oct 14 '15 at 16:22
You might have to `setup.py install` with manual prefix args. Also does this problem happen with other packages, doses something common like `requests` have this same behavior? — ThorSummoner, Oct 14 '15 at 19:23
could you check shebang (!#) of the pip in the virtualenv (/home/user/.virtualenvs/fred/bin/pip)? if you have multiple python interpreters, could you please create virtualenv with explicit python interpreter? (-p parameter to mkvirtualenv) last - are you sure that there is no pip in the new virtualenv? that you need to install new one? — Jerzyk, Jul 01 '16 at 10:08
Try deactivating and reactivating (without rebuilding the virtual env). Perhaps the activation when you create the environment is not the same as normal activation. — falsePockets, May 23 '17 at 03:44
Have you run `source fred/bin/activate` to activate the virtualenv? After you run this your shell should show `(fred)$` instead of just `$` before commands. — Ahmouse, Jun 26 '18 at 08:32
I'm also confused as to why you installed `pip` a second time, when `virtualenv` already installs it for you. @jalanb — Ahmouse, Jun 26 '18 at 08:37
What happens when you run `pip3.4 install itsdangerous`? What happens When you run `/home/user/.virtualenvs/fred/bin/pip install itsdangerous`? — FazeL, Jun 26 '18 at 12:26
@Skilledfire I used [workon](https://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html#controlling-the-active-environment) instead of activate. My shell should not show `(fred)$`, because I control my prompt. The `$ which python pip` command show which virtualenv is active — jalanb, Jun 26 '18 at 18:08
@Skilledfire I installed pip a second time because I did not know that `virtualenv` already installed it for me. TIL. — jalanb, Jun 26 '18 at 18:09
@FazeL I have no idea what the result of running `v 3.4` commands are, but they are not relevant to this question. The question includes the output from `$ which pip; pip install itsdangerous` — jalanb, Jun 26 '18 at 18:12
Have you tried using a different virtualenv? Try creating a new one somewhere else, then use the pip install by virtualenv. Then after that try doing what you did in this virtualenv, and compare the results. @jalanb — Ahmouse, Jun 26 '18 at 20:32
No longer have access to `this virtualenv` for the comparison, and the question is years old. But re-running the command on current system gives no problem. "current system" is `macOS`. The "question system" was `CentOS`. — jalanb, Jun 27 '18 at 02:49

score 1 · Answer 1 · answered Jul 02 '18 at 13:14

1

Probably your are calling the system level pip and not the virtualenv pip.

You should activate your virtualenv before:

. bin/activate

answered Jul 02 '18 at 13:14

Fabio Caccamo

1,871
19
21

I had [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html#workon) installed, so the `activate` is covered by `$ workon fred` – jalanb Jul 04 '18 at 23:31
ok, I never used virtualenvwrapper, but I suggest you to try [pipenv](https://github.com/pypa/pipenv) – Fabio Caccamo Jul 05 '18 at 10:29
I have tried [pipenv](https://stackoverflow.com/search?tab=votes&q=%5bpython%5d%20pipenv), and [alternatives](https://stackoverflow.com/questions/41573587/what-is-the-difference-between-venv-pyvenv-pyenv-virtualenv-virtualenvwrappe/41573588#41573588), but prefer [local rules](https://gitlab.com/jalanb/jab/blob/master/src/bash/cde.sh#L186) for controlling virtual envs these days As I have 200 characters left: `pipenv` is not as good as `requests`, because it is too opinionated (about what a host project should be). But, unfortunate to see developers ganging up on invalid characters again. – jalanb Jul 05 '18 at 16:14

Why is pip, inside a virtualenv, writing to /usr/lib?

1 Answers1