1

I have a Python project that runs in a docker container and I am trying to convert to a multistage docker build process. My project depends on the cryptography package. My Dockerfile consists of:

# Base                                                                          
FROM python:3.6 AS base                                                         

RUN pip install cryptography                                                    

# Production                                                                    
FROM python:3.6-alpine                                                          

COPY --from=base /root/.cache /root/.cache                                      

RUN pip install cryptography \                                                  
        && rm -rf /root/.cache                                                  

CMD python

Which I try to build with e.g:

docker build -t my-python-app .

This process works for a number of other Python requirements I have tested, such as pycrypto and psutil, but throws the following error for cryptography:

Step 5/6 : RUN pip install cryptography         && rm -rf /root/.cache
 ---> Running in ebc15bd61d43
Collecting cryptography
  Downloading cryptography-2.1.4.tar.gz (441kB)
Collecting idna>=2.1 (from cryptography)
  Using cached idna-2.6-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography)
  Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting six>=1.4.1 (from cryptography)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting cffi>=1.7 (from cryptography)
  Downloading cffi-1.11.5.tar.gz (438kB)
    Complete output from command python setup.py egg_info:

        No working compiler found, or bogus compiler options passed to
        the compiler from Python's standard "distutils" module.  See
        the error messages above.  Likely, the problem is not related
        to CFFI but generic to the setup.py of any Python package that
        tries to compile C code.  (Hints: on OS/X 10.8, for errors about
        -mno-fused-madd see http://stackoverflow.com/questions/22313407/
        Otherwise, see https://wiki.python.org/moin/CompLangPython or
        the IRC channel #python on irc.freenode.net.)

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-uyh9_v63/cffi/

Obviously I was hoping not to have to install any compiler on my production image. Do I need to copy across another directory other than /root/.cache?

Jon G
  • 1,656
  • 3
  • 16
  • 42
  • I think I have used the cryptography package in some capacity while using PyJWT and some custom encryption algorithm, and the last I remember, I did not need to compile it twice, anywhere. – Divij Sehgal Mar 02 '18 at 18:19
  • Besides, the ideal way of doing dependency installation would be keeping all application library dependencies in a requirements file, copying it into the container from within the ```Dockerfile``` and then running ```pip install -r``` on it within the same Dockerfile – Divij Sehgal Mar 02 '18 at 18:22
  • The idea behind having a multi stage docker build (see https://lekum.org/post/multistage-dockerfile/) is that you can build these libraries using a full image like `python3.6`, then copy the artifacts to a lightweight image to be run. The method I am outlining shouldn't build the library twice, if I can correctly copy across the artifacts. – Jon G Mar 05 '18 at 09:08
  • hi @JonG So did you figure this out? – StackEdd Jun 14 '18 at 08:38
  • No, sorry. In cases like this, I tend to just build using slim rather than alpine. – Jon G Jun 15 '18 at 11:03

2 Answers2

1

There is no manylinux wheel for Alpine, so you need to compile it yourself. Below is pasted from documentation on installation. Install and remove build dependencies in the same command to only save the package to the docker image layer.

If you are on Alpine or just want to compile it yourself then cryptography requires a compiler, headers for Python (if you’re not using pypy), and headers for the OpenSSL and libffi libraries available on your system.

Alpine Replace python3-dev with python-dev if you’re using Python 2.

$ sudo apk add gcc musl-dev python3-dev libffi-dev openssl-dev

If you get an error with openssl-dev you may have to use libressl-dev.

Docs can be found here

Love Tätting
  • 221
  • 2
  • 6
  • I thought I had already built the library in the base docker stage. I just wanted to copy across the results. Similar to what they did here: https://lekum.org/post/multistage-dockerfile/ ... That isn't possible? – Jon G Mar 05 '18 at 09:04
  • Sorry for being late to the show. So, the problem is that cffi tries to compile during in its setupscript to see that a compiler is there.It literally tries to compile a string, much like a configure script works. I don't know if one can circumvent this check through pip. To make everything work as intended on alpine (python:3.6 is debian based) I wouldn't try to simply copy the packages (i.e. compiled libraries and site-packages) and rather do as in my answer. In alpine; install compiler && install python dependencies && remove compiler. The compiler will not be committed to image. – Love Tätting Mar 16 '18 at 21:19
  • hi @LoveTätting could you provide the link to the documentation. – StackEdd Jun 14 '18 at 08:40
  • @StackEdd I have added a link – Love Tätting Jun 16 '18 at 20:49
  • 1
    I follwed this. The installation works. However, when I run the application it says: Unable to import module 'lambda_function': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory. Any idea about this one :( ? – Ankur Bhatia Jul 07 '20 at 20:21
0

I hope, my answer will be useful.

  1. You should use --user option for cryptography installing via pip in base stage. Example: RUN pip install --user cryptography. This option means, that all files will be installed in the .local directory of the current user’s home directory.
  2. COPY --from=base /root/.local /root/.local, because cryptography installed in /root/.local.

Thats all. Full example docker multistage 

# Base                                                                          
FROM python:3.6 AS base                                                         

RUN pip install --user cryptography

# Production
FROM python:3.6-alpine

COPY --from=base /root/.local /root/.local

RUN pip install cryptography \
        && rm -rf /root/.cache

CMD python
Denis Sukhoverkhov
  • 90
  • 1
  • 6