0

I am trying to deploy a streamlit app with Docker. I need scipy, so I tried installing in a separate RUN command and with requirements.txt. The image builds fine, but when I run the app I get the following error:

File "main.py", line 14, in load_models
pickle.load(open("model.pk"), "rb")

ModuleNotFoundError: No module named 'scipy.sparse._csr'

Dockerfile:

FROM python:3.6-slim

WORKDIR /app

RUN apt-get update
RUN apt-get install \
    'ffmpeg'\
    'libsm6'\
    'libxext6'  -y

RUN apt-get install rustc -y

RUN pip install numpy
RUN apt update && apt install -y build-essential && rm -rf /var/lib/apt/lists/*
RUN pip install scikit-image matplotlib more_itertools

COPY ./requirements.txt .

RUN pip install -r requirements.txt

COPY . .

RUN pip freeze > requiremets.txt

RUN cat requiremets.txt

EXPOSE 8000
EXPOSE 8501

CMD ["python", "main.py"]

requirements.py:

scipy<=1.5.4
classla
transformers
pickle-mixin
streamlit
scikit-learn

The Python version is 3.6.15.

This is the output of the pip show command in interactive mode in the container:

altair                4.1.0
argon2-cffi           21.3.0
argon2-cffi-bindings  21.2.0
async-generator       1.10
attrs                 22.2.0
backcall              0.2.0
backports.zoneinfo    0.2.1
bleach                4.1.0
blinker               1.5
cachetools            4.2.4
certifi               2022.12.7
cffi                  1.15.1
charset-normalizer    2.0.12
classla               1.1.0
click                 8.0.4
commonmark            0.9.1
cycler                0.11.0
dataclasses           0.8
decorator             4.4.2
defusedxml            0.7.1
entrypoints           0.4
filelock              3.4.1
gitdb                 4.0.9
GitPython             3.1.18
huggingface-hub       0.4.0
idna                  3.4
imageio               2.15.0
importlib-metadata    4.8.3
importlib-resources   5.4.0
ipykernel             5.5.6
ipython               7.16.3
ipython-genutils      0.2.0
ipywidgets            7.7.3
jedi                  0.17.2
Jinja2                3.0.3
joblib                1.1.1
jsonschema            3.2.0
jupyter-client        7.1.2
jupyter-core          4.9.2
jupyterlab-pygments   0.1.2
jupyterlab-widgets    1.1.2
kiwisolver            1.3.1
lxml                  4.9.2
MarkupSafe            2.0.1
matplotlib            3.3.4
mistune               0.8.4
more-itertools        8.14.0
nbclient              0.5.9
nbconvert             6.0.7
nbformat              5.1.3
nest-asyncio          1.5.6
networkx              2.5.1
notebook              6.4.10
numpy                 1.19.5
obeliks               1.1.6
packaging             21.3
pandas                1.1.5
pandocfilters         1.5.0
parso                 0.7.1
pexpect               4.8.0
pickle-mixin          1.0.2
pickleshare           0.7.5
Pillow                8.4.0
pip                   21.2.4
prometheus-client     0.16.0
prompt-toolkit        3.0.36
protobuf              3.19.6
ptyprocess            0.7.0
pyarrow               6.0.1
pycparser             2.21
pydeck                0.6.2
Pygments              2.14.0
Pympler               1.0.1
pyparsing             3.0.9
pyrsistent            0.18.0
python-dateutil       2.8.2
pytz                  2022.7.1
pytz-deprecation-shim 0.1.0.post0
PyWavelets            1.1.1
PyYAML                6.0
pyzmq                 25.0.0
regex                 2022.10.31
reldi-tokeniser       1.0.2
requests              2.27.1
rich                  12.6.0
sacremoses            0.0.53
scikit-image          0.17.2
scikit-learn          0.24.2
scipy                 1.5.4
semver                2.13.0
Send2Trash            1.8.0
setuptools            57.5.0
six                   1.16.0
smmap                 5.0.0
streamlit             1.10.0
terminado             0.12.1
testpath              0.6.0
threadpoolctl         3.1.0
tifffile              2020.9.3
tokenizers            0.12.1
toml                  0.10.2
toolz                 0.12.0
torch                 1.10.2
tornado               6.1
tqdm                  4.64.1
traitlets             4.3.3
transformers          4.18.0
typing_extensions     4.1.1
tzdata                2022.7
tzlocal               4.2
urllib3               1.26.14
validators            0.20.0
watchdog              2.3.0
wcwidth               0.2.6
webencodings          0.5.1
wheel                 0.37.0
widgetsnbextension    3.6.2
zipp                  3.6.0
vvvvv
  • 25,404
  • 19
  • 49
  • 81
Veneta
  • 33
  • 5
  • 1
    Have you tried running the docker image in interactive mode? Might help to run some pip commands to see what's going on (`pip show scipy`), as well as running `which pip && which python` to see if there might be any discrepancies. – Tae Hyung Kim Feb 25 '23 at 20:26
  • @TaeHyungKim I tried running in interactive mode. The show command shows the correct version of scipy (1.5.4) and it seems to be compatible with the python version (3.6.15). – Veneta Feb 25 '23 at 21:04
  • 1
    I think you might be missing a package. Running `import scipy.sparse` in docker interactive mode works, but `import scipy.sparse._csr` doesn't. I'm not too familiar with the package. – Tae Hyung Kim Feb 25 '23 at 21:28
  • 1
    This also seems relevant: https://stackoverflow.com/questions/72631150/joblib-load-error-no-module-named-scipy-sparse-csr – Tae Hyung Kim Feb 25 '23 at 21:30

1 Answers1

0

Turns out the error was because of conflicting Python versions with the imported models. It was fixed by using a newer version.

reference: Joblib.load error: No module named 'scipy.sparse._csr'

Veneta
  • 33
  • 5