0

What I am trying to do?

Install all dependencies mentioned in requirements.txt using downloaded wheel files i.e. offline installation of packages in Docker

What I have done?

By following this thread I managed to download all my wheels into a wheelhouse folder using mkdir wheelhouse && pip download -r requirements.txt -d wheelhouse and I created a compressed tarball wheelhouse.tar.gz containing all my downloaded .whl files along with a requirements.txt

When I try to install the wheels locally (outside Docker) using pip install -r wheelhouse/requirements.txt --no-index --find-links wheelhouse, it works!

But when I run the same in Docker, it doesn't with the following error:

Processing ./wheelhouse/beautifulsoup4-4.8.2-py3-none-any.whl 
ERROR: Could not find a version that satisfies the requirement blis==0.4.1 (from -r ./wheelhouse/requirements.txt (line 2)) (from versions: none)
ERROR: No matching distribution found for blis==0.4.1 (from -r ./wheelhouse/requirements.txt (line 2))

While actually, the wheel for blis 0.4.1 is present in my wheelhouse directory.

Can anyone please help me identify why it doesn't run on Docker and runs on local?

Dockerfile

FROM python:3

COPY . /app
WORKDIR /app

RUN tar -zxf ./wheelhouse.tar.gz 

RUN pip install -r ./wheelhouse/requirements.txt --no-index --find-links ./wheelhouse

Screenshot of wheelhouse directory:

c

Vasu Mistry
  • 781
  • 2
  • 6
  • 18
  • 1
    Your `blis` version is made for macosx_10_6_intel, etc. Your docker container probably doesn't run MacOSX. You'll need the correct whl file for the architecture of your docker container instead of the one from your parent environment. I'm not sure how to do this, but since docker caches the immediate steps when building containers, copying the requirements file first, then installing dependencies, then copying your own code (i.e. the thing that changes) effectively makes the package installation offline (since it'll only re-run when requirements.txt changes). – MatsLindh Mar 25 '20 at 09:45
  • I was just about to write this down too! Thanks @MatsLindh for your quick response. I'll try this – Vasu Mistry Mar 25 '20 at 09:46
  • Can you please tell me how can I obtain the architecture of my docker container? All I have done is install Docker from the .dmg file (https://docs.docker.com/docker-for-mac/install/) – Vasu Mistry Mar 25 '20 at 10:24
  • Usually it'll be linux amd64, but in general I advise that you install and build the requirements as a separate step inside your container. That will be far more stable than having to manually handle the build steps outside of your actual container being built. – MatsLindh Mar 25 '20 at 10:48
  • I was able to install the packages, it depends on the architecture as well as python version! e.g. package for python 3.7 won't work if your docker is running 3.8. @MatsLindh How do you recommend creating the 2 steps - package installation and container buillt. Something on the lines of 2 docker files? – Vasu Mistry Mar 25 '20 at 11:25
  • In general you only have a single Dockerfile, but copy the `requirements.txt` file in first, then use pip on that file - that way you get a) the correct versions for python version and architecture b) get it automagically rebuilt when the requirements change c) it'll be cached _unless_ the requirements change d) it won't depend on any process outside of the Dockerfile (i.e. when another developer might have to build the image themselves, everything is done automagically). – MatsLindh Mar 25 '20 at 11:36
  • But as long as the offline installation is a requirement, you'll have to have some sort of manual process in place anyway - i.e. a `update_dependencies.sh` script that does everything outside of the Dockerfile and build the libraries that should be included. That process will have to be online, and will have to run each time requirements.txt changes, so I'm not sure what it'll actually buy you compared to Docker caching the images for each step anyway. – MatsLindh Mar 25 '20 at 11:37
  • Hint: https://stackoverflow.com/a/55125446/7976758; found in https://stackoverflow.com/search?q=%5Bpip%5D+download+platform – phd Mar 25 '20 at 12:10

0 Answers0