ValueError: "Requirement" object has no field "use_pep517" when installing Python packages in a Docker Image

Question

I'm having a problem installing Python dependencies using pipenv inside a Dockerfile.

Dockerfile:

RUN pip install --upgrade pip
RUN pip install pipenv

# Copy dependencies source code
WORKDIR /projects

# Copy project source code
WORKDIR /projects/source
COPY ./projects/source .

# Install packages
RUN pipenv install --system --deploy

We used to build the image successfully with the following pipenv dependencies:

# Result of pip install pipenv
Successfully installed certifi-2023.5.7 distlib-0.3.6 filelock-3.12.2 pipenv-2023.7.11 platformdirs-3.9.1 setuptools-68.0.0 virtualenv-20.24.0 virtualenv-clone-0.5.7

But now, we're running into this error when we do a pipenv install --system --deploy in our Docker image:

# Result when executing pipenv install --system --deploy
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pipenv/cli/command.py", line 233, in install
    do_install(
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 170, in do_install
    do_init(
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 777, in do_init
    do_install_dependencies(
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 455, in do_install_dependencies
    batch_install(
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 596, in batch_install
    batch_install_iteration(
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 538, in batch_install_iteration
    _cleanup_procs(project, procs, failed_deps_queue, retry=retry)
  File "/usr/local/lib/python3.10/site-packages/pipenv/routines/install.py", line 651, in _cleanup_procs
    dep.use_pep517 = True
  File "/usr/local/lib/python3.10/site-packages/pipenv/vendor/requirementslib/models/common.py", line 18, in __setattr__
    raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
ValueError: "Requirement" object has no field "use_pep517"

I did notice a slight change of the distlib version when Docker tries to install the pipenv, but not sure if this is the problem

Successfully installed certifi-2023.5.7 distlib-0.3.7 filelock-3.12.2 pipenv-2023.7.11 platformdirs-3.9.1 setuptools-68.0.0 virtualenv-20.24.0 virtualenv-clone-0.5.7

I tried upgrading my local pipenv and rebuilding the Pipfile.lock, but it still gives the same error when I tried building it inside a Docker image.

Answer 1

I hit the same issue for several of my Python projects built in Docker. It looks like it's an issue with PyYAML. Here's the other post where I saw the fix:

Docker-compose no longer building image (AttributeError: cython_sources)

I went with two different workarounds for different projects. In one project, I just upgraded packages that were all working again. In a different project I ended up having to switch back to PyYAML 5.3.1. The projects are internal only so I am not concerned about security much.

One thing to note is that trying to install the specific package that is failing (I tried them one by one until I found the culprit) with pip instead of pipenv will produce a more specific error message which led me to the linked answer.

Hope this helps!

Answer 2

I hit the same issue in previously fine CI pipeline. For me, regenerating the lock file fixed it. You can regenerate the lock file using:

pipenv lock

Answer 3

I had a similar problem and the response from Cristhian Ulloa (https://stackoverflow.com/a/76724894/201657) helped me out, so thought I'd share my circumstances.

In my Pipfile I was installing schemachange==3.4.1 which depended upon PyYAML==5.4.1. I changed to schemachange==3.5.3 which installed PyYAML==6.0.1, ran pipenv lock to update Pipfile.lock. git pushed the two changed files ... and my GitHub Actions pipeline succeeded whereas it was failing beforehand.

Answer 4

I had the same error, but it was not caused by PyYAML at all. As @cristhian-ulloa suggested, you may need to dig in to find the offending package. In my case, it was mysqlclient@2.2.0.

I found it by running install in verbose mode, which lists packages as they're installed.

pipenv install --verbose

...
24.05 Collecting markuppy==1.14 (from -r 
24.05 /tmp/pipenv-c3w01jdq-requirements/pipenv-k9rwlph0-hashed-reqs.txt (line 16))
24.05 
24.05   Using cached MarkupPy-1.14.tar.gz (6.8 kB)
24.05 
24.06   Preparing metadata (setup.py): started
24.06 
24.88   Preparing metadata (setup.py): finished with status 'done'
24.88 
24.88 
25.08 Collecting mysqlclient==2.2.0 (from -r 
25.08 /tmp/pipenv-c3w01jdq-requirements/pipenv-k9rwlph0-hashed-reqs.txt (line 17))
25.08 
25.08   Using cached mysqlclient-2.2.0.tar.gz (89 kB)
25.08 
25.32 ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml
25.32 Traceback (most recent call last):
...

Looking at the output you see mysqlclient-2.2.0 is the last package before the error. I then install that alone with pip

pipenv run pip install mysqlclient

...
7.159 Collecting mysqlclient
7.301   Downloading mysqlclient-2.2.0.tar.gz (89 kB)
7.357      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB 1.9 MB/s eta 0:00:00
7.393   Installing build dependencies: started
11.18   Installing build dependencies: finished with status 'done'
11.18   Getting requirements to build wheel: started
11.81   Getting requirements to build wheel: finished with status 'error'
11.82   error: subprocess-exited-with-error
11.82   
11.82   × Getting requirements to build wheel did not run successfully.
11.82   │ exit code: 1
11.82   ╰─> [24 lines of output]
11.82       Trying pkg-config --exists mysqlclient
11.82       /bin/sh: 1: pkg-config: not found
11.82       Command 'pkg-config --exists mysqlclient' returned non-zero exit status 127.
11.82       Trying pkg-config --exists mariadb
11.82       /bin/sh: 1: pkg-config: not found
11.82       Command 'pkg-config --exists mariadb' returned non-zero exit status 127.
11.82       Traceback (most recent call last):
11.82         File "/root/.local/share/virtualenvs/code-_Py8Si6I/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
11.82           main()
...

Here I see the root issue is that pkg-config is missing. Which led me to this SO answer: https://stackoverflow.com/a/76560124/5275148

I install pkg-config in the container and all is well.

Basically, the issue could be PyYAML, but might be something completely different. I hope this helps you step through and find the culprit.

Answer 5

In m case it's the backports.zoneinfo==0.2.1 version with a Python 3.10.4 and 3.11.4 installation. However, I just don't use it with the package it came with, as that zoneinfo is now in the standard library and I don't plan to use Python < 3.9.