2

Edit: I'm going to close this question as the reason its happening is different from my original assumption, and it's clearer to ask the question anew:

Pip installs packages in the wrong directory with virtualenv

The accepted answer doesn't directly answer the original question, but is a very useful overview.

Based on discussion below the issue is that even after

$ source ~/PycharmProjects/Practice/venv/bin/activate 
$ pip install numpy

numpy is installed in /usr/local/lib/python2.7/site-packages
What could the reason for this be?

Original:

Using Python on OS X via Homebrew:

I've been trying much of the day to sort this out but either I get a must supply either home or prefix/exec-prefix -- not both error, or the package I try to install goes into totally the wrong place:

$ pip3 --version
pip 18.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
$ cd venv
$ pip3 install numpy
..... [snip with following error:]
"must supply either home or prefix/exec-prefix -- not both")

Using this hint

$ pip3 install numpy -t .

Then I get a new error, 

`Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/.../pip-install-0fvveq3v/package/'

Searching around SO gives various possibilities involving pip install setuptools. but pip install throws the above error or installs in the wrong place. i.e. the solution involves something that's causing the error in the first place.

I tried to use the Python.org installer but it didn't install pip at all. (The custom installer showed the option checked but with size zero).

Gazzer
  • 4,524
  • 10
  • 39
  • 49
  • 2
    you're not in a virtualenv here though, did you exit it by mistake – gold_cy Feb 11 '19 at 11:51
  • 2
    You need to `source venv/bin/activate` instead of `cd venv` – Matias Cicero Feb 11 '19 at 11:55
  • Can you elaborate: if I go to `$ cd ~/PycharmProjects/Practice3/venv/bin $ ./pip3 install numpy` it still installs in the wrong place: "Installing collected packages: numpy" but in "/usr/local/lib/python2.7/" – Gazzer Feb 11 '19 at 11:55
  • 1
    @Gazzer just do `source ~/PycharmProjects/Practice3/venv/bin/activate`. you don't need to go there yourself. After that, your `pip` command will point to the binary inside the virtual environment, so just do `pip install numpy`. – Matias Cicero Feb 11 '19 at 11:56
  • `$ source ~/PycharmProjects/Practice3/venv/bin/activate` `$ pip install numpy` `Target directory /usr/local/lib/python2.7/site-packages/numpy-1.16.1.dist-info already exists.` It still doesn't go into the venv. It ends up in python2.7 – Gazzer Feb 11 '19 at 11:59
  • 1
    How did you create the virtual environment? – Matias Cicero Feb 11 '19 at 12:01
  • It's automatically generated with PyCharm, and using the Preferences in there doesn't work either. It's probably the same issue. I'd be happy to create the virtualenv by scratch if necessary. (I've never used Python before, and this issue is maddening - although now I've properly understood the point of the `source` command. – Gazzer Feb 11 '19 at 12:03
  • Of course, I'd love to do this `python3 -m pip install --user virtualenv` but I'm struggling to deal with the `"must supply either home or prefix/exec-prefix -- not both")`error mentioned above – Gazzer Feb 11 '19 at 12:09
  • 1
    Once you `activate` the `pip` command will do the right thing, no options required. – tripleee Feb 11 '19 at 12:19
  • @tripleee Unfortunately, that's not the case here (I think this is why it's taken me so long to get to grips with the issue). When I run `$ source ~/PycharmProjects/Practice3/venv/bin/activate` followed by `pip install numpy` numpy is installed in `/usr/local/lib/python2.7/site-packages` which makes no sense at all as I'm running version 3 anyway. I'm wordering if there is any config file that could be causing this issue. – Gazzer Feb 11 '19 at 12:22

1 Answers1

2

An introductory overview is available in this nice tutorial. Here is a good summary with more detail. But, if you renamed or moved the virtual env dir after its creation, it could break it. Create a new one from scratch: $ cd ~/PycharmProjects; python3 -mvenv newenv ; Activate: $ source newenv/bin/activate ; Install something: $ pip install colorama (same as pip3 install only if venv activated); Check: ls ~/PycharmProjects/newenv/lib/python3*/site-packages ; Deactivate: $ deactivate

Then you could try this solution for Pycharm: how to associate a virtual environment with a python project in pycharm. PyCharm indeed comes bundled with virtualenv which could have been customized, please look into Pycharm-specific resources: creating virtual environments and installing packages in Pycharm.

If you have installed PyPI's mainstream virtualenv, by default it will create new environments with the python interpreter that virtualenv was installed with. But it's possible to specify an alternate Python Interpreter upon a new env creation: $ virtualenv -p python3.7 newenvname

Regarding the error DistutilsOptionError: must supply either home or prefix - please check this and this for solutions. Homebrew'ed mappings between python and pip are described here. The normal pip install --user is disabled in homebrewed Python, but there are workarounds. MacOS system Python doesn't provide pip, but it can installed, reinstalled or upgraded for any specific python version manually. Original non-brewed installers are also available for all Python versions: https://www.python.org/downloads/mac-osx/

By default there's no pip.conf, but it can be created by hand to customize things. All possible pip.conf locations (per-user, per-venv, and global/system-wide, and how they override each other) are listed here. If someone faces an issue, they could use pip config list command to see their active configuration, or locate pip.conf and find it.

Finally, you may want to ensure you aren't using pip against macOS' system python. Shell commands such as $ brew info python, which pip, which pip3, pip3 -V, which python3 can help you see what you actually use. Since the macOS default $PATH used to be /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin, stock macOS binaries (including python) may take precedence over some homebrew'ed installations (including python). If so, a custom PATH could be exported via the ~/.bashrc if required.

stop.climatechange.now
  • 3,923
  • 18
  • 27
  • Thank you, but still the same issue. `pip install colorama` puts the packages inside `/usr/local/lib/python2.7/site-packages/` Is there any invisible home setting that could cause Python to put it here instead? – Gazzer Feb 11 '19 at 12:52
  • @Gazzer You have installed python with `brew`? – stop.climatechange.now Feb 11 '19 at 12:56
  • Yes, with Brew. – Gazzer Feb 11 '19 at 12:57
  • @Gazzer Then this is a [known issue](https://stackoverflow.com/questions/24257803/distutilsoptionerror-must-supply-either-home-or-prefix-exec-prefix-not-both) with `brew`. – stop.climatechange.now Feb 11 '19 at 12:58
  • That explains the `"must supply either home or prefix/exec-prefix -- not both")` issue but not the pip install issue. (Also that issue is not just Brew - it occurs with the regular python install) – Gazzer Feb 11 '19 at 13:00
  • @Gazzer `$ pip -V; pip3 -V` ? It's possible to reinstall pip, once the prefix is fixed. – stop.climatechange.now Feb 11 '19 at 13:10
  • `$ curl https://bootstrap.pypa.io/get-pip.py | sudo /Users/{your user name}/{path to python}/bin/python` (specify the exact location of your virtual Python environment) – stop.climatechange.now Feb 11 '19 at 13:13
  • I think there must be some other issue here. PyCharm installs its own virtual environment and when I try to use that within the Terminal using ` source venv/bin/activate` things are installed in the python2.7 folder. I would prefer to avoid having to go through all these steps each time I make a new project. Since the issue is different from how I first asked it, I'm going to post a new question. I'll post the link here soon. – Gazzer Feb 11 '19 at 14:57
  • https://stackoverflow.com/questions/54633456/pip-installs-packages-in-the-wrong-directory-with-virtualenv – Gazzer Feb 11 '19 at 15:07
  • I'm accepting this as the answer as it's really useful. See my note in the question itself. – Gazzer Feb 12 '19 at 02:05