40

I am sure it is not network issue. Some of my machine install packages using pip is very fast while some other machine is pretty slow, from the logs, I suspect the slow is due to it will compile the package, I am wondering how can I avoid this compilation to make the pip installation fast. Here's the logs from the slow pip installation.

Collecting numpy==1.10.4 (from -r requirements.txt (line 1))
  Downloading numpy-1.10.4.tar.gz (4.1MB)
    100% |████████████████████████████████| 4.1MB 95kB/s
Requirement already satisfied (use --upgrade to upgrade): wheel==0.26.0 in ./lib/python2.7/site-packages (from -r requirements.txt (line 2))
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy ... -
done
  Stored in directory: /root/.cache/pip/wheels/66/f5/d7/f6ddd78b61037fcb51a3e32c9cd276e292343cdd62d5384efd
Successfully built numpy
zjffdu
  • 25,496
  • 45
  • 109
  • 159
  • 1
    Notice ` Stored in directory: /root/.cache/pip/wheels/66/f5/d7/f6ddd78b61037fcb51a3e32c9cd276e292343cdd62d5384efd`: it's cacheing the build output (which it'll reuse) -- I imagine you occasionally get a slow installation when you miss this cache (especially for really slow-to-compile packages like numpy) – anthony sottile Feb 02 '16 at 02:58
  • Some packages need to be compiled. I don't think there's any way around that. – ChrisGPT was on strike Feb 02 '16 at 03:10
  • it may be a network issue, as pypi is using multiple servers (CDN) for delivery, other issues, that may be an issue: for some of the machines it is finding wheels (prebuild) packages and for some it is compiling from source – Jerzyk Jul 01 '16 at 09:40
  • @Chris Sometimes using an older version of Python will get around building dependencies. – dstromberg May 31 '22 at 15:59

5 Answers5

26

The slowness is due to compilation indeed. But there is now the manylinux tag. Which allows the installation of pre-compiled distributions. See for example the PyPI page of numpy to see if a manylinux package is provided for your Python version.

Update (2021-06)

Since this answer received some attention lately, there are now many manylinux tags for precompiled packages (no pun intended).

code_onkel
  • 2,759
  • 1
  • 16
  • 31
  • 1
    how would I indicate to pip that I want to use the wheel? – vidstige Jan 15 '18 at 13:25
  • 10
    I don't know whether you can force pip to do so. For my environments, pip chose the precompiled packages automatically (if applicable, I guess). But I suppose you need a sufficiently new version of pip. Try `pip install --upgrade pip setuptools wheel` before `pip install numpy`. – code_onkel Jan 15 '18 at 13:30
  • 1
    @code_onkel I'm concerned by pip performance on Windows. Afaik pip has a cache for packages and wheels nowadays. Still installing `PyGLM numpy numba Pillow` takes up to 25 seconds with cache. With `venv-update` it's almost instant with their `pip-faster` tool but it's limited to pip 18+ and `venv-update` is not quite maintained. Do you please know others tools out there like `pip-faster`and more reliable or any tips to speed up pip through configuration? – khelkun Feb 17 '22 at 09:18
  • 1
    @khelkun I did not test on windows, but it seems that even with caching, pip checks PyPI for each package. Also, 25 seconds does not seem terribly bad, depending on your hardware. Did you rule out network and disk latency? – code_onkel Feb 19 '22 at 14:32
12

In case anyone was having the network issue and landed on this page like me:

I noticed slowness on my machine because pip install would get stuck in network calls while trying to create socket connections (sock.connect()). As discussed here, this can happen when the host supports IPv6 but your network doesnt. As instructed here, I checked if this was true by disabling IPv6 on my Ubuntu machine as follows :

sysctl net.ipv6.conf.all.disable_ipv6=1

I was no longer hanging in network calls after this change.

However, I am not sure if this is a sustainable solution since we are all slowly moving to IPv6.

Harsh Verma
  • 529
  • 6
  • 10
  • worked for me - this is how you do it in macOS: https://protonvpn.com/support/how-to-disable-ipv6-protocol-on-macos/ – ababoolal Feb 12 '23 at 21:40
3

For me, I was running into this issue with pip 22.0.4, Ubuntu 20.04.4 LTS. I was installing tensorflow-gpu, which already takes too much time, but the pip was unusually very slow. The above solutions didn't make any sense to me, so I did the following:

  • Do not run pip commands with sudo.
  • apt-update && apt-upgrade
  • Reboot the server/computer

I know it doesn't seem to be a permanent solution, but it fixed the issue for me.

Singh
  • 504
  • 4
  • 15
1

If you are using Anaconda, try updating pip, this solved it for me.

Singh
  • 504
  • 4
  • 15
Salo7ty
  • 475
  • 5
  • 18
-44

Command

Instead of (too slow to complete)

python -m pip install numpy

This worked (fast as supposed to be)

pip install numpy

Check

python

import numpy as np

(it shouldn't give any errors)

ynn
  • 3,386
  • 2
  • 19
  • 42
Ramy Rabie
  • 1
  • 2