Creating a docker container that runs Ubuntu with Python 3.6+ and Pip

Question

I'm trying to install Python 3.6 or above with pip in an docker container that runs Ubuntu. I've tried quite a few things with no success

FROM ubuntu:18.04
RUN apt update
RUN apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa
RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN apt install python3.8 -y
RUN apt install pip

RUN pip install auto-sklearn
RUN pip install pandas

ADD test.py /

CMD [ "python", "./test.py" ]

This returns "Unable to locate package pip." I tried removing "apt install pip" incase Python 3.8 comes with it, but it gives me the error: "pip: not found."

FROM ubuntu:18.04
RUN apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get update
RUN install python3-pip
RUN pip install auto-sklearn
RUN pip install pandas

ADD test.py /

CMD [ "python", "./test.py" ]

This installs pip, but auto-sklearn requires Python version 3.6 or higher and this installs a lower version. Auto-sklearn requires Linux as well which is why I'm using "FROM ubuntu" rather than "FROM python" cause "FROM python" seems to build a container on whatever native OS is running on the computer building the container, which for me is Windows.

Make sure to use the right binaries, like `python3.8` and `pip3.8` or `python3,8 -m pip` all the time. You can also consider to use a newer Ubuntu version or a `python` baseimage. — Klaus D., Jan 19 '21 at 22:27
@KlausD. My binaries look right, no? I also tried "Ubuntu:latest" and get the same errors and as I mentioned in the question I think the python base images boot using the same OS that my computer is running cause I get the auto-sklearn error exclusive to installing on non-Linux systems — , Jan 19 '21 at 22:29
No, you used the simple `pip` and `python` without version number multiple times. — Klaus D., Jan 19 '21 at 22:33
@KlausD. okay, what exactly are you referring to when you say "binaries?" — , Jan 19 '21 at 22:36
@KlausD. I mispoke I guess I didn't try the top commands with "ubuntu:latest" because that seemed to work. If you want to post your answer and allow me to edit it and add the full Dockerfile I used, I'll give you the upvote and answer vote. — , Jan 19 '21 at 22:54

score 4 · Answer 1 · answered Sep 28 '21 at 12:55

I see two consecutive issues here, so let's address them accordingly:

Issue 1: missing pip in the Ubuntu image

This returns "Unable to locate package pip." I tried removing "apt install pip" incase Python 3.8 comes with it, but it gives me the error: "pip: not found."

That's right. If you inspect the contents of the /usr/bin directory of the pulled image, you will notice that there is no pip or pip3 there. So RUN ln -s /usr/bin/pip3 /usr/bin/pip line in your Dockerfile does nothing. Even when python3.6 gets installed in the container (after calling apt install software-properties-common -y), you do not get pip with it.

Solution: install pip

The following commands can be used to install python3.6 binary and the corresponding pip:

RUN apt-get update
RUN apt-get install python3-pip

This installs both python3.6 and pip3 in the /usr/bin directory of your ubuntu:18/04 container.

Issue 2: auto-sklearn requires python >= 3.7

Even if you manage to get both python3.6 and pip for python3.6, installation of auto-sklearn might still fail with the following error:

    RuntimeError: Python version >= 3.7 required.

This is because some of the dependencies (e.g. ConfigSpace package) require python version >= 3.7.

Solution:

This answer explains how to install pip for python3.8 on Ubuntu: https://stackoverflow.com/a/63207387/15043192

You can follow it or get both pip and python3.8 installed using the following sequence:

Install python3.8:

RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install python3.8

Install python3.6 and pip for python3.6

RUN apt install python3-pip

Now if you execute python3.6 -m pip --version in the container, you would get something like (the version of pip might be different):

pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

Install pip for python3.8

Note: here we use previously installed pip for python3.6 to install pip for python3.8. Do not ask me why :-)

RUN python3.8 -m pip install pip --upgrade

Install auto-sklearn

RUN python3.8 -m pip install auto-sklearn

Note: the command above might also install pandas package among other dependencies of auto-sklearn.

Create a symbolic link to python3.8

This would change the default

RUN ln -s /usr/bin/python3.8 /usr/bin/python

Now if you execute python -m pip --version in the container, you would get something like (the version of pip might be different):

pip 21.2.4 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)

Grand finale:

In the end, your Dockerfile should look like this:

FROM ubuntu:18.04

RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install python3.8
RUN apt install python3-pip
RUN python3.8 -m pip install auto-sklearn
RUN python3.8 -m pip install pandas
RUN ln -s /usr/bin/python3.8 /usr/bin/python

ADD test.py /

CMD [ "python", "./test.py" ]

NB

To avoid messing around with different versions of python and pip, you might want to have a look into virtual environments.

`ERROR: failed to solve: process "/bin/sh -c add-apt-repository ppa:deadsnakes/ppa" did not complete successfully: exit code: 127` — alper, Jun 17 '23 at 18:30