36

This question is as much a question about my particular problem (which I sort of found a work-around, so it's not a burning issue) as it is about the general process I am using.

Setup (the part that works):

I have Python 2.7.9 installed locally on my Ubuntu 14.04, and I have a virtualenv in which I am running it. Everything is very much separated from the "system" Python, which I am not touching.

The part I did:

It all started well enough, with my Python installed and all libraries running. For example, I also pip installed numpy 1.10.1, it compiled for a while, then it worked just fine.

The problem:

The problem is that for reasons beyond my control, I had to rebuild the python with ucs4 enabled, that is I installed it using

./configure --enable-unicode=ucs4

After doing this, I also uninstalled all libraries and reinstalled them using pip. However, it seems that the numpy library was not properly uninstalled because it installed instantly this time, and when I tried to import numpy into my new Python, I got an error message indicating that the numpy was compiled with the ucs2-enabled Python.

This hypothesis is pretty solid, since I tried then to pip install numpy==1.9.3. The installation once again took a long time, and it produced a numpy version that works on the new ucs4 enabled Python.

Now, my question:

How can I get the numpy uninstallation process to delete all traces of the old numpy?

Edit:

I also tried to manually remove numpy by deleting it from my virtualenv site-packages directory. After deleting, import numpy returned an ImportError as expected. I then reinstalled it (pip install numpy) and it came back with the same ucs2-related error.

Edit 2:

The full sys.path seen by my virtualenv Python is

['',
 '/home/jkralj/.virtualenvs/work/lib/python27.zip',
 '/home/jkralj/.virtualenvs/work/lib/python2.7',
 '/home/jkralj/.virtualenvs/work/lib/python2.7/plat-linux2',
 '/home/jkralj/.virtualenvs/work/lib/python2.7/lib-tk',
 '/home/jkralj/.virtualenvs/work/lib/python2.7/lib-old',
 '/home/jkralj/.virtualenvs/work/lib/python2.7/lib-dynload',
 '/usr/local/lib/python2.7.9/lib/python2.7',
 '/usr/local/lib/python2.7.9/lib/python2.7/plat-linux2',
 '/usr/local/lib/python2.7.9/lib/python2.7/lib-tk',
 '/home/jkralj/.virtualenvs/work/lib/python2.7/site-packages']

Also, it might be important to mention that the /usr/local/lib/python2.7.9/ installation of python does not have numpy installed.

Community
  • 1
  • 1
5xum
  • 5,250
  • 8
  • 36
  • 56
  • @ali_m I tried that, but it did not work. – 5xum Nov 12 '15 at 16:24
  • @ali_m no, check the edit I wrote. It did not work in that it did uninstall numpy, but reinstalling it brought the same error back. Also, `which python` prints the virtualenv python – 5xum Nov 12 '15 at 17:16
  • @ali_m Your hypothesis does not explain the fact that installing numpy 1.9.3 works just fine. – 5xum Nov 12 '15 at 17:19
  • @ali_m I edited my question some more. – 5xum Nov 12 '15 at 17:24
  • @ali_m As I said, "I then reinstalled it (`pip install numpy`) and it came back with the same error." – 5xum Nov 12 '15 at 17:40
  • @ali_m It worked! `pip install -v numpy` showed that it was istalling numpy out of a cached `whl` file it had build with the old python (`vcs2` enabled). Running `pip install numpy --no-cache-dir` forced `pip` to install anew, and it recompiled numpy into something that actually works. – 5xum Nov 12 '15 at 17:44
  • @ali_m Thank you so much for all your help. If you post any kind of answer to this question, I will gladly accept it... – 5xum Nov 12 '15 at 17:46
  • 1
    No, I think you should answer this yourself (you have access to more of the details anyway). One other suggestion: you should definitely clear the pip cache to avoid running into similar issues with other modules that have compiled extensions. Usually this lives in `~/.cache/pip/`. There might also a temporary build directory in `/tmp/` ([see here](http://stackoverflow.com/q/9510474/1461210)). – ali_m Nov 12 '15 at 20:08
  • @ali_m I will answer it myself then. You don't get any rep, but thank you for your help anyway. You saved me a lot of trouble. – 5xum Nov 12 '15 at 21:20

2 Answers2

53

You can use --no-binary and --ignore-installed to rebuild a package as follows

pip install --user --force-reinstall --ignore-installed --no-binary :all: PackageName
Frank Breitling
  • 942
  • 1
  • 10
  • 27
  • 1
    `--user` is superfluous here since OP uses virtualenv. – Ivan Anishchuk Jun 29 '17 at 10:30
  • Regarding `--force-reinstall`, `pip help install` says: "When **upgrading**, reinstall all packages even if they are already up-to-date." – x-yuri Feb 03 '18 at 00:02
  • I have found `--ignore-installed` to cause troubles as it will only install new files, not update existing ones which leaves an inconsistent state. `--force-reinstall` on pip 21 does work as expected – Flamefire Apr 28 '21 at 12:04
  • 2
    Note that `:all:` is the argument to `--no-binary`, so the order of the flags matters. – AI0867 Mar 09 '22 at 09:55
26

The problem is solved by pip uninstalling numpy (or any other troublesome package), then running

pip install numpy --no-cache-dir

to prevent pip from simply taking the cached installation and repeating it.

5xum
  • 5,250
  • 8
  • 36
  • 56
  • 5
    `--no-cache-dir` did not force a _recompilation_, it just made pip download the package again. `--no-binary :all:` forced a recompilation (running of `setup.py`). – Praveen Jun 12 '17 at 06:04
  • @Praveen in my case, --no-cache-dir also caused a recompilation (as a consequence of downloading the package again, I guess) – 5xum Jun 12 '17 at 06:06
  • It's possible that this behaviour varies somewhat from system to system. I've had pip do different things in Ubuntu and Fedora, and also do different things when run in a virtualenv. But `--no-cache-dir` didn't cut it for me :-/ – Praveen Jun 12 '17 at 06:09
  • 1
    `--no-cache-dir --force-reinstall` worked for me with pip 23.1.2 on macOS Catalina 10.15.7. – Stefan Schmidt May 09 '23 at 00:06