0

I'm trying to build an Docker image which should run a python script, which needs numpy, scipy, pandas and google-cloud-bigquery.

Since this image is build for an armv7 architecture it's a pain to install numpy, scipy and pandas directly (it takes too long and finally it breaks). So I decided to use Miniconda and used the packeges for Raspberry Pi. That worked fine (installation can be completet during image build).

Now I'm trying to install the google packages google-crc32c==1.1.2 and google-cloud-bigquery. With pip this is possible and the image is build properly. But if I run a container with this Image it is always restarting and gives me this error log:

File "/usr/src/app/bigquery.py", line 1, in <module>
    from google.cloud import bigquery
ImportError: No module named 'google'

I think I have to install the google packages with conda but there are no packages for armv7 architecture available:

google-cloud-bigquery package on Anaconda.org: https://anaconda.org/search?q=google+bigquery

google-crc32c package on Anaconda.org: https://anaconda.org/search?q=google-crc32c

Is there a possibility to install those google packages with Miniconda for armv7 architecture? Or is another way possible to install numpy, scipy and pandas without using miniconda (but not installing them directly)?

Thank you for any help!

Dockerfile:

FROM python:3.7-buster

WORKDIR /usr/src/app

ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"

COPY main_prog.py bigquery.py requirements.txt ./

RUN wget http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-armv7l.sh
RUN mkdir /root/.conda
RUN /bin/bash Miniconda3-latest-Linux-armv7l.sh -b
RUN rm -f Miniconda3-latest-Linux-armv7l.sh \
    && echo "Running $(conda --version)"

RUN wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh
RUN chmod +x Berryconda3-2.0.0-Linux-armv7l.sh ./Berryconda3-2.0.0-Linux-armv7l.sh
   
RUN conda list \
    && conda config --add channels rpi \
    && conda install python=3.6 -y\
    && conda install openblas blas -y\
    && conda install numpy -y\
    && conda install pandas -y\
    && conda install scipy -y

RUN pip install --upgrade pip

RUN pip install "google-crc32c==1.1.2"
RUN pip install google-cloud-bigquery

CMD ["python", "main_prog.py"]
Ohaiomundo
  • 31
  • 7
  • Have you seen this [question with similar issue](https://stackoverflow.com/questions/36183486/importerror-no-module-named-google). Not the same architecture, but maybe some of ideas will work. – vitooh Sep 22 '21 at 07:55
  • Yes I've seen this, but unfortunately that doesn't helpt me. I could install the packages with wheels from [link](https://www.piwheels.org). So I don't install them with Miniconda anymore. – Ohaiomundo Sep 23 '21 at 07:48

1 Answers1

1

I couldn't find I way to install all packages with Miniconda.

But it was possible for me to install them directly with wheels from piwheels. Therefor I had to add a pip.conf file in "/etc" dirctory.

content of pip.conf:

[global]
extra-index-url=https://www.piwheels.org/simple

In addition I had to install libatlas-base-dev. I only could do this by adding an URL deb http://ftp.de.debian.org/debian buster main (like it's recommended here) to my sources.list in "/etc/apt/" directory.

content of sources.list:

# deb http://snapshot.debian.org/archive/debian/20210902T000000Z buster main
deb http://deb.debian.org/debian buster main
# deb http://snapshot.debian.org/archive/debian-security/20210902T000000Z buster/updates main
deb http://security.debian.org/debian-security buster/updates main
# deb http://snapshot.debian.org/archive/debian/20210902T000000Z buster-updates main
deb http://deb.debian.org/debian buster-updates main
deb http://ftp.de.debian.org/debian buster main

Dockerfile:

FROM python:3.7-buster

WORKDIR /usr/src/app

COPY main_prog.py bigquery.py requirements.txt pip.conf sources.list ./

RUN mv ./pip.conf /etc \
    && export PIP_CONFIG_FILE=/etc/pip.conf

RUN mv ./sources.list /etc/apt/

RUN apt-get update \
    && apt-get upgrade -y

RUN apt-get install libatlas-base-dev -y
RUN pip3 install --upgrade pip

RUN pip3 install numpy \
    && pip3 install scipy \
    && pip3 install pandas \
    && pip3 install google-crc32c \
    && pip3 install google-cloud-bigquery

CMD ["python", "main_prog.py"]
Ohaiomundo
  • 31
  • 7