7

I am trying to use pip-tools to manage a venv (as in python -m venv .venv) environment. The freshly-activated environment has nothing but pip-tools initially:

> pip list
Package   Version
--------- -------
Click     7.0
pip       19.3.1
pip-tools 4.2.0
six       1.13.0

I created a requirements/main.in file with just:

numpy
matplotlib

Running pip-compile --upgrade --build-isolation --generate-hashes --output-file requirements/main.txt requirements/main.in gives me this warning:

# WARNING: The following packages were not pinned, but pip requires them to be
# pinned when the requirements file includes hashes. Consider using the --allow-unsafe flag.
# setuptools==41.6.0        # via kiwisolver
The generated requirements file may be rejected by pip install. See # WARNING lines for details.

As warned, pip install --upgrade -r requirements/main.txt rejects the operation with:

ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    setuptools from https://files.pythonhosted.org/packages/d9/de/554b6310ac87c5b921bc45634b07b11394fe63bc4cb5176f5240addf18ab/setuptools-41.6.0-py2.py3-none-any.whl#sha256=3e8e8505e563631e7cb110d9ad82d135ee866b8146d5efe06e42be07a72db20a (from kiwisolver==1.1.0->-r requirements/main.txt (line 11))

So now my predicament is: should I use --allow-unsafe? What are the implications of doing this? I tried it and found that the generated requirements file has it pinned to that particular version, which kiwisolver (I guess a transitive dependency of numpy/matplotlib) needs right? But why is this unsafe?

If I understand it correctly, I can keep my generated requirements for as long as I want, but then - whenever I decide to update - I can re-run pip-compile and create a new requirements file will be fresh and it may have a new kiwisolver that does not have this restriction, right?

What is the reason pip-tools is asking ME to make this decision? Why is this potentially unsafe and what examples are there where one would NOT want to use --allow-unsafe?

A related question is: can I specify to "--allow-unsafe" ONLY for setuptools? It seems to be a parameter of pip-compile which is an all-or-nothing approach. Can I just flag the particular one as "ok to pin"? I would like to be warned again if some OTHER case arises so I can assess whether that's ok or not?

Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
Alexandros
  • 2,097
  • 20
  • 27

1 Answers1

8

So now my predicament is: should I use --allow-unsafe? What are the implications of doing this?

Yes, you should. This option allows you to pin in requirements.txt the following packages: distribute, pip, and setuptools. If you don't care, go ahead!

But why is this unsafe?

AFAIK, those packages could be considered unsafe for the following reasons:

  • Changing the setuptools may cause conflicts with pip ( distribute is the legacy wrapper of setuptools, and it's deprecated since 2013).
  • Changing pip could break pip-tools itself or your system pip.

The --allow-unsafe option most likely will be deprecated in the near future, see discussions in pip-tools' and pip's issue-trackers.

phoenix
  • 7,988
  • 6
  • 39
  • 45
Albert Tugushev
  • 1,504
  • 13
  • 25
  • Thanks for including the link to those issue trackers. This seems to be like a "theoretically correct bootstrapping concern" which in practice everyone will want to ignore and just allow the "unsafe" behavior. – Alexandros Nov 15 '19 at 00:08
  • 2
    I'm inclined to think this flag is there for historical reasons, which may be not relevant nowadays. – Albert Tugushev Nov 15 '19 at 04:36