How do I create a docker container with both Python 2 and 3 available in Jupyter Notebooks?

Question

I am trying to create a docker container that has anaconda and supports Jupyter notebooks with both python 2 and 3. I created a container based on the official anaconda python 3 container like so:

FROM continuumio/anaconda3:latest

WORKDIR /app/
COPY requirements.txt /app/

RUN pip install --upgrade pip && \
    pip install -r requirements.txt

Once on the container, I am able to get python 2 and 3 working with Jupyter notebooks by entering the following commands:

conda create -y -n py2 python=2.7
conda activate py2
conda install -y notebook ipykernel
ipython kernel install --user
conda deactivate

Then when I go back to base and run jupyter kernelspec list I see:

(base) root@1683850aacf0:/app# jupyter kernelspec list
Available kernels:
  python2    /root/.local/share/jupyter/kernels/python2
  python3    /root/.local/share/jupyter/kernels/python3

and when I open a jupyter notebook server I see both python 2 and 3 options. This is the state that I would like to end up in. I tried to turn all these into docker commands like so:

RUN    conda create -y -n py2 python=2.7
RUN    conda activate py2
RUN    conda install -y notebook ipykernel
RUN    ipython kernel install --user
RUN    conda deactivate

but running the command to activate or deactivate (RUN conda activate py2) a conda environment gives me an error:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

Adding RUN conda init bash before the commands doesn't change the error message.

Also, based on this SO question I tried:

RUN conda create -y -n py3 python=3.7 ipykernel
RUN conda create -y -n py2 python=2.7 ipykernel

but after I build and enter the container I only see the python 3 environment:

(base) root@b301d8ab5f1e:/app# jupyter kernelspec list
Available kernels:
  python3    /opt/conda/share/jupyter/kernels/python3

I can activate py2 and see that kernel, but not both:

(py2) root@b301d8ab5f1e:/app# jupyter kernelspec list
Available kernels:
  python2    /opt/conda/envs/py2/share/jupyter/kernels/python2

What else should I try?

EDIT:

I tried specifying the shell as Adiii suggested with the following:

FROM continuumio/anaconda3:latest

WORKDIR /app/
COPY requirements.txt /app/

RUN pip install --upgrade pip && \
    pip install -r requirements.txt

ENV BASH_ENV ~/.bashrc
SHELL ["/bin/bash", "-c"]
RUN    conda create -y -n py2 python=2.7
RUN    conda activate py2
RUN    conda install -y notebook ipykernel
RUN    ipython kernel install --user
RUN    conda deactivate

This allows the container to build but for some reason there was no python 2.7 environment:

(base) root@31169f698f14:/app# jupyter kernelspec list
Available kernels:
  python3    /root/.local/share/jupyter/kernels/python3
(base) root@31169f698f14:/app# conda info --envs
# conda environments:
#
base                  *  /opt/conda
py2                      /opt/conda/envs/py2

(base) root@31169f698f14:/app# conda activate py2
(py2) root@31169f698f14:/app# jupyter kernelspec list
Available kernels:
  python3    /root/.local/share/jupyter/kernels/python3

see this https://github.com/cclauss/Python2-and-Python3-in-Docker/blob/master/Dockerfile — LinPy, Oct 18 '19 at 05:12
Interesting approach, but this doesn't look like it includes Anaconda — jss367, Oct 18 '19 at 06:28

score 0 · Answer 1 · answered Oct 18 '19 at 02:12

From this issue, you need sepcify the SHELL directive in the Dockerfile, like SHELL ["/bin/bash", "-c"]. The problem could be the fact that the default shell in RUN command is sh.

This is similar to the solutions above, but avoids some of the boilerplate in every RUN command:

ENV BASH_ENV ~/.bashrc
SHELL ["/bin/bash", "-c"]

Then something like this should work as expected:

RUN conda activate my-env && conda info --envs

Or, to set the environment persistently (including for an interactive shell) you could:

RUN echo "conda activate my-env" >> ~/.bashrc

Dockerfile

FROM continuumio/anaconda3:latest

WORKDIR /app/

RUN pip install --upgrade pip
ENV BASH_ENV ~/.bashrc
SHELL ["/bin/bash", "-c"]
RUN    conda create -y -n py2 python=2.7
RUN    conda activate py2
RUN    conda install -y notebook ipykernel
RUN    ipython kernel install --user
RUN    conda deactivate

I tried this and it built but didn't provide python 2. I'll post the output in my question — jss367, Oct 18 '19 at 02:34
@jss367 the answer addresing this issue "CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'Currently supported shells are:" if there is another problem you can another question instead of updating each time the question:) — Adiii, Oct 18 '19 at 06:52

score 0 · Answer 2 · answered Feb 20 '20 at 23:07

This was what ended up working:

RUN conda env create -f py2_env.yaml
RUN conda env create -f py3_env.yaml
RUN /bin/bash -c "conda init bash && source /root/.bashrc && conda activate py2 && conda install -y notebook ipykernel && ipython kernel install --user && conda deactivate"
RUN /bin/bash -c "conda init bash && source /root/.bashrc && conda activate py3 && conda install -y notebook ipykernel && ipython kernel install --user && conda deactivate"

How do I create a docker container with both Python 2 and 3 available in Jupyter Notebooks?

2 Answers2