56

I'm trying to install numpy in a docker container based on Alpine 3.1. I'm using the following Dockerfile:

FROM alpine:3.1
RUN apk add --update make cmake gcc g++ gfortran
RUN apk add --update python py-pip python-dev
RUN pip install cython
RUN pip install numpy

This runs fine until pip install numpy when I get the following error:

error: Command "gcc -fno-strict-aliasing -Os -fomit-frame-pointer -DNDEBUG -Os -fomit-frame-pointer -fPIC -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -c build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.c -o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.o" failed with exit status 1

easy_install-2.7 numpy gives the same error.

Are there any config/installation steps I'm missing?

BMW
  • 42,880
  • 12
  • 99
  • 116
TDN169
  • 1,397
  • 2
  • 13
  • 31

9 Answers9

48

I've been having a bit of trouble with this myself and, long story short, I would encourage you to ask if it's really worth the hassle. Numpy is enormous when you start adding things to the stack like pandas, gpus, and scipy so the benefit of building it on alpine is limited, the savings over using Debian, Arch, or even Ubuntu are relatively modest when 500MB of your space is on this library anyway.

That having been said, I threw together an image that does it. I needed as build-time dependencies musl-dev, linux-headers, and g++. I also wound up needing to add openblas from edge for something later in the stack so it's possible that some dependencies from that are required too. But I believe just adding the three former libraries with

apk --no-cache add musl-dev linux-headers g++

should be sufficient to prevent the gcc error you are getting. You can view the image at https://hub.docker.com/r/o76923/alpine-numpy-stack/

James Endicott
  • 589
  • 4
  • 3
  • 1
    Worked for me on the `python:3-alpine` image even without `linux-headers`. Thanks! – Norrius Jun 01 '18 at 14:05
  • My answer is a bit old to this question now. The `py-numpy-dev` package in community edge seems to work now. – James Endicott Jun 02 '18 at 22:44
  • 2
    James, any chance you have the exact image size differences? – Glen Thompson Oct 11 '18 at 21:50
  • Another advantage of the Debian / Arch / Ubuntu approach is that those distros can often use the prebuilt `manylinux` wheels off of PyPI, so there's a decent chance that you wouldn't even _need_ build-time dependencies. – Josh Kelley May 06 '21 at 17:39
43

If you don't necessary need to install numpy from pypi, you could install it from alpine repositories. Package is named py-numpy and is in testing repository, see here. Minimal Dockerfile example that works for me

FROM alpine:3.2
ADD repositories /etc/apk/repositories
RUN apk add --update python python-dev gfortran py-pip build-base py-numpy@community

Content of repositories file

http://dl-cdn.alpinelinux.org/alpine/v3.2/main
@community http://dl-cdn.alpinelinux.org/alpine/edge/community
Sharpless512
  • 3,062
  • 5
  • 35
  • 60
ziky
  • 864
  • 8
  • 13
  • 4
    This is working, but unfortunately that's numpy for python 2.7. We need a version for 3.5 – leonprou Jun 21 '16 at 12:49
  • 2
    As an update, you need to switch to `@community` everywhere instead of `@testing`, e.g. `py-numpy@community` and `@community http://dl-cdn.alpinelinux.org/alpine/edge/community` – Rikki Schulte Sep 16 '16 at 16:49
  • I have tried everything in this post including @James-Endicott image below and the install crashes invariably around NumPy. Also tried https://github.com/WattyAB/docker.alpine.numerical-python which unfortunately didn't work. Will just live with a 650MB image.. – cardamom Jun 02 '17 at 15:06
  • 3
    These days this should be `py3`. A minimal install with lapack is `apk add python3-dev py3-numpy lapack` with no other packages required (`gfortran` dependency is pulled automatically). – Qualia May 26 '20 at 18:31
  • But why this is a problem in the first place? I.e., why there is no suitable numpy wheel for alpine – banderlog013 Aug 17 '21 at 21:12
  • no pip on alpine, no wheels, so has to build from source which takes ages for large packages with diverse dependencies on things like blas gcc..... – ldgorman Mar 08 '22 at 10:25
  • @Qualia Your suggestion is interesting, but sadly doesn't work on iPad because `py3-numpy` assumes SSE and SSE2 extensions which are not present on iPad CPU. – Mark Setchell Jul 24 '23 at 08:56
15

A package is now available in the Alpine repository: py3-numpy. But you won't be able to use it straightaway.

py3-numpy installs libraries into /usr/lib/python3.8/site-packages directory but the default Python module path does not use it:

$ docker run -it python:3.8-alpine sh
/ # apk add --update --no-cache py3-numpy
/ # python
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> import sys
>>> sys.path
['', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']

This can be fixed by setting the $PYTHONPATH environment variable to the path of the site-packages in /usr/lib:

FROM python:3.8-alpine

RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.8/site-packages
confusius
  • 571
  • 1
  • 4
  • 14
Faylixe
  • 478
  • 6
  • 12
13

This one is about 311MB according to my docker images:

FROM python:3.6-alpine
RUN apk add g++ 
RUN pip install numpy

(Meanwhile python:3.6 is ~900MB by itself)

Have you tried NOT having gcc installed? It might be conflicting? Not sure. This one worked for me as a minimal numpy installation and wanted to share.

Multihunter
  • 5,520
  • 2
  • 25
  • 38
5

Alpine is built with musl, which is incompatible with python wheels. That means that either all the dependencies should be installed via apk or they should be compiled manually. For a smoother experience with python pypi dependencies it looks more optimal to use debian, cropped to the minimum size (python:slim) as a starting point:

FROM python:slim
CMD pip install numpy

123Mb

This approach is way simpler than the accepted answer and the resulting image is more compact than in the other answers.

Antony Hatchkins
  • 31,947
  • 10
  • 111
  • 111
2

Try this:

RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas
Gil Margolin
  • 1,959
  • 20
  • 21
1

With optimizations such as removing build dependencies after build and removing unneeded tests (they are here because we're building the module, not just installing it):

FROM frolvlad/alpine-python3

RUN apk add --no-cache \
        --virtual=.build-dependencies \
        g++ file binutils \
        musl-dev python3-dev cython && \
    apk add libstdc++ openblas && \
    ln -s locale.h /usr/include/xlocale.h && \
    pip install numpy && \
    rm -r /root/.cache && \
    find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
    find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
    rm /usr/include/xlocale.h && \
    apk del .build-dependencies

Resulting size ~157MB.

funnydman
  • 9,083
  • 4
  • 40
  • 55
  • Interesting! Would you comment why `xlocale.h` is needed? I see other recipes calling for e.g. `gfortran` as well and I'm curious what's the **authoritative** dependency set. – Dima Tisnek Feb 25 '21 at 00:35
  • @DimaTisnek, I don't actually remember why it was needed. But the flow is simple: try to build as small as possible a workable image if it does not build then add missing dependencies and try again. In the end, I'm trying not to use alpine, most of the time slim version is enough. – funnydman Feb 25 '21 at 10:41
  • Good one :) It seem `gfortran` is only needed for local development, to run tests. – Dima Tisnek Feb 26 '21 at 00:58
  • 1
    ref for xlocale: https://github.com/numpy/numpy/pull/8367 – Dima Tisnek Feb 26 '21 at 00:59
  • Works for me! Took ~15 mins to compile. Yet I've found a faster (and slightly more compact: 125MB vs 127MB) [method](https://stackoverflow.com/a/70942710/237105). – Antony Hatchkins Feb 01 '22 at 15:02
0

Standard PyPI wheels don’t work on Alpine

  • Why? Most Linux distributions use the GNU version (glibc) of the
    standard C library that is required by pretty much every C program,
    including Python. But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel
    support.

  • Most Python packages these days include binary wheels on PyPI,
    significantly speeding install time. But if you’re using Alpine Linux you need to compile all the C code in every Python package that you
    use.

  • found a solution to use this:

    RUN apk --update add gcc libc-dev libffi-dev build-base freetype-dev libpng-dev openblas-dev

But Alpine builds are slower & the image is bigger.

  • In addition, while in theory the musl C library used by Alpine is mostly compatible with the glibc used by other Linux distributions, in practice the differences can cause problems.

All in all its better to use python:slim instead of python:alpine because its not worth the hassle

Tasnim Tabassum
  • 21
  • 2
-1

simply use a docker image with numpy pre-installed: https://hub.docker.com/r/adreeve/python-numpy/

Psyndrom Ventura
  • 131
  • 6