Can't recreate Conda environment in docker

Question

I created a conda environment from a fresh installation of miniconda3.

After that I exported it and this is the content of the file (my only extra install was flask):

name: myenv
channels:
  - defaults
dependencies:
  - ca-certificates=2018.03.07=0
  - certifi=2018.11.29=py37_0
  - click=7.0=py37_0
  - flask=1.0.2=py37_1
  - itsdangerous=1.1.0=py37_0
  - jinja2=2.10=py37_0
  - libcxx=4.0.1=hcfea43d_1
  - libcxxabi=4.0.1=hcfea43d_1
  - libedit=3.1.20170329=hb402a30_2
  - libffi=3.2.1=h475c297_4
  - markupsafe=1.1.0=py37h1de35cc_0
  - ncurses=6.1=h0a44026_1
  - openssl=1.1.1a=h1de35cc_0
  - pip=18.1=py37_0
  - python=3.7.1=haf84260_7
  - readline=7.0=h1de35cc_5
  - setuptools=40.6.2=py37_0
  - sqlite=3.26.0=ha441bb4_0
  - tk=8.6.8=ha441bb4_0
  - werkzeug=0.14.1=py37_0
  - wheel=0.32.3=py37_0
  - xz=5.2.4=h1de35cc_4
  - zlib=1.2.11=h1de35cc_3
prefix: /Users/rossid/miniconda3/envs/phadmin

now what I want, is to recreate this environment in a docket image so I created this Dockefile

FROM continuumio/miniconda3
ADD * myappdir/
RUN conda env create -f /myappdir/environment.yml

but it will fail with:

Step 1/5 : FROM continuumio/miniconda3
 ---> d3c252f8727b
Step 2/5 : ADD * myappdir/
 ---> Using cache
 ---> 2afbf5ea75bd
Step 3/5 : RUN conda env create -f /myappdir/environment.yml
 ---> Running in 7f916bd46979
Solving environment: ...working... failed

ResolvePackageNotFound: 
  - tk==8.6.8=ha441bb4_0
  - ncurses==6.1=h0a44026_1
  - markupsafe==1.1.0=py37h1de35cc_0
  - readline==7.0=h1de35cc_5
  - zlib==1.2.11=h1de35cc_3
  - openssl==1.1.1a=h1de35cc_0
  - xz==5.2.4=h1de35cc_4
  - libcxxabi==4.0.1=hcfea43d_1
  - libcxx==4.0.1=hcfea43d_1
  - libffi==3.2.1=h475c297_4
  - sqlite==3.26.0=ha441bb4_0
  - python==3.7.1=haf84260_7
  - libedit==3.1.20170329=hb402a30_2

why is this happening? If I try to do the same to create another environment it works. If I remove the build version, some dependencies are resolved (I mean the third coordinate in dependencies).

I tried to add more channels like conda-forge, but nothing.

Also my .condarc file is empty.

Does anyone know how to fix this?

perhaps [this may help](https://github.com/datitran/object_detector_app/issues/41), as I have faced a similar issue in the past. You could remove build version and version version although some apps may work differently if a different version is installed — d_kennetz, Dec 17 '18 at 17:14
Is it the same OS to generate the environment.yml and within the Docker image? — darthbith, Dec 19 '18 at 15:34
some of these libraries are platform dependent like: libcxxabi and libcxx these are for OSX, but in Linux they are not available. Some of these libs only work if I put them in the PIP section. I'd like to make an environment.yml that is cross platform but I don't know how... — gotch4, Dec 20 '18 at 16:14
@gotch4 I don't think cross-platform-compatible conda environment files exist at the moment, and that's because conda itself isn't fully cross-platform-compatible. Correct me if I'm wrong, internet. — mjbeyeler, Jun 25 '19 at 12:43

score 3 · Answer 1 · answered Jan 13 '21 at 10:11

I had a similar problem and I find multiple ways to solve it. The main problem with your approach is conda is not platform independent, so will force the environments to use pip.

1. Conda Like Solution

Change your my_env.yml so that all the dependencies apart from pip goes under the pip dependency. Notice that the syntax is different when you move under the pip.

For instance:

name: myenv
channels:
  - defaults
dependencies:
   - pip=18.1
   - pip:
     - wheel==0.32.3

Then go to your Dockerfile and add the following line:

RUN conda env update -n base --file myenv.yml

2. Good old Pip way

Export your conda environment into a pip requirements file as at this answer

conda install pip
pip freeze > requirements.txt

Then go to your Docker file and add the following line:

RUN python -m pip install -r requirements.txt

god why does package management suck so much in python? – Adam Hughes Feb 17 '22 at 17:23 — Adam Hughes, Feb 17 '22 at 17:23

score 0 · Answer 2 · answered Nov 23 '22 at 20:31

The docker build fails because your input yml file includes platform-specific build constraints. For example:

ResolvePackageNotFound: 
  - tk==8.6.8=ha441bb4_0
  - ncurses==6.1=h0a44026_1
  - markupsafe==1.1.0=py37h1de35cc_0
  - readline==7.0=h1de35cc_5
  - zlib==1.2.11=h1de35cc_3
  - openssl==1.1.1a=h1de35cc_0
  - xz==5.2.4=h1de35cc_4
  - libcxxabi==4.0.1=hcfea43d_1
  - libcxx==4.0.1=hcfea43d_1
  - libffi==3.2.1=h475c297_4
  - sqlite==3.26.0=ha441bb4_0
  - python==3.7.1=haf84260_7
  - libedit==3.1.20170329=hb402a30_2

Those packages contain platform specific hashs (e.g. ha441bb4_0). Basically, you are trying to install packages from the OS platform on a linux platform. That's why berkay's answer would work for most of the use case. A simpler way to solve this problem is add from-history argument while exporting your conda environment.

conda env export -f env_explicit.yml --from-history

This argument will just includes packages, which you explicitly specified during installation. This argument will also ignore any platform-specific dependencies.

And your new Dockerfile will looks like following:

FROM continuumio/miniconda3
ADD * myappdir/
RUN conda env create -f /myappdir/env_explicit.yml

Reference: conda fails to create environment from yml

Can't recreate Conda environment in docker

2 Answers2