28

I found that in different folders, sometimes pip install will build wheel which takes a lot of time, while sometimes it doesn't. I'm not sure why and how to control that.

My command: bin/python -m pip install -r ../requirements.txt (due to the !# shebang line-length limitation, so I don't use pip directly)

The output without building a wheel (just takes a few seconds):

Collecting numpy==1.10.4 (from -r ../requirements.txt (line 1))
Installing collected packages: numpy
Successfully installed numpy-1.10.4

The output with building wheel (take at least 2 minutes)

Collecting numpy==1.10.4 (from -r ../requirements.txt (line 1))
  Downloading numpy-1.10.4.tar.gz (4.1MB)
    100% |████████████████████████████████| 4.1MB 92kB/s
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy ... done
  Stored in directory: /root/.cache/pip/wheels/66/f5/d7/f6ddd78b61037fcb51a3e32c9cd276e292343cdd62d5384efd
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.10.4

The contents of requirements.txt:

numpy==1.10.4
Rob Bednark
  • 25,981
  • 23
  • 80
  • 125
zjffdu
  • 25,496
  • 45
  • 109
  • 159

4 Answers4

22

Today I encountered a problem where a package wasn't being installed properly because it turns out that its build process generates incorrect wheel packages, even though direct installation works just fine.

I did a bit of poking around, and it turns out that as of now (pip == 8.1.2), there isn't a direct way to control whether or not pip will try to build a wheel out of a given package. I found the relevant source code, and apparently, the wheel build process is used if and only if:

  • the wheel module is importable
  • a cache directory is in use

As a result of that logic, one can indirectly disable pip's use of wheel-based builds by passing --no-cache-dir on the install command line.

0 _
  • 10,524
  • 11
  • 77
  • 109
Simon Ruggier
  • 411
  • 3
  • 5
  • 2
    Looks like this logic has changed quite a bit (now at ``pip==19``). Code can now be traced to https://github.com/pypa/pip/blob/cf722df4f9d7f8ee2d62e53ef2354ea65e932170/src/pip/_internal/wheel_builder.py#L48. Unfortunately, there doesn't appear to be a convenient flag to turn off the wheel building functionality (other than uninstalling wheel) whilst keeping wheel installations for packages already with wheels. – pelson Jan 14 '20 at 12:35
  • 5
    Actually, you can be specific about which packages shouldn't generate wheels by setting the ``--no-binary={package_name}`` apparently... – pelson Jan 14 '20 at 12:47
  • 1
    Pip also has the `--only-binary` and `--prefer-binary` arguments. – cowlinator Feb 10 '21 at 20:06
10

This depends on whether your package is a pure python package (without the need to compile anything, just copy the files somewhere) or a package which also includes c source code (in which case a compilation is necessary and a compiler is called and executed, which takes longer).

http://pythonwheels.com/

You may also want to have a look at the wheel docu:

http://wheel.readthedocs.org/en/latest/

tfv
  • 6,016
  • 4
  • 36
  • 67
  • 13
    Whether or not `numpy` is a pure python package would describe whether *it* would cause python to build a wheel, but that doesn't explain why sometimes it builds the wheel and sometimes it doesn't, as the question shows. – Cory Klein Sep 18 '17 at 19:56
5

I got the answer, it is just the first time that the wheel will be build, after that, it will read from cache

zjffdu
  • 25,496
  • 45
  • 109
  • 159
0

When I install slither-analyzer using pip3 on my Mac, I got a similar error.

Building wheels for collected packages: pysha3
  Building wheel for pysha3 (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]

The solution is to reinstall your Xcode tools.

$ xcode-select --install
$ xcode-select --reset
saneryee
  • 3,239
  • 31
  • 22