1

So I am working in an external organization's GCP project, and the policy is that there is no public internet access available. I want to use some Python modules that are not contained in the DL containers that are available for Vertex, so the only option is to build and push a docker container that installs those modules for me, then use it to create a jupyter notebook.

Here's the catch: I am trying to install a certain module version that requires at least Python 3.9, and that's where I am stuck now. I've accomplished building with Python 3.9 using this as a reference in my Dockerfile:

FROM gcr.io/deeplearning-platform-release/base-cpu:latest
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    add-apt-repository -y ppa:deadsnakes/ppa && \
    apt-get update && \
    apt install -y python3.9
RUN pip install gower
RUN pip install kmodes
RUN pip install ray
RUN pip install prince==0.9.0

But I am getting this error during build:

> [6/6] RUN pip install prince==0.9.0:
#0 1.325 ERROR: Ignored the following versions that require a different python version: 0.8.0 Requires-Python >=3.9,<4.0; 0.8.1 Requires-Python >=3.9,<4.0; 0.8.2 Requires-Python >=3.9,<4.0; 0.8.3 Requires-Python >=3.9,<4.0; 0.9.0 Requires-Python >=3.9,<4.0
#0 1.325 ERROR: Could not find a version that satisfies the requirement prince==0.9.0 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.5.2, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.7.0, 0.7.1)
#0 1.326 ERROR: No matching distribution found for prince==0.9.0

So I am assuming it is not finding the Python version previously installed and thus fails to build. I guess I am missing a couple of lines and/or instructions that would allow me to properly build and push this to Container Registry.

Any feedback is greatly appreciated!

  • Your image probably already has some python installed. Either remove it (e.g. `apt-get remove python3.8`) or call `python3.9 -m pip install ...` to reference exact python. – STerliakov Mar 31 '23 at 16:06
  • @SUTerliakov yes, the base 'gcr.io/deeplearning-platform-release/base-cpu' has 3.7.X installed, including some other data science packages. Let's say I remove python 3.7 first, what happens to those packages when I install 3.9? Would I need to reinstall all of them? If so, then including a requirements.txt would be the best practice, I assume. – elAndrez3000 Mar 31 '23 at 17:59
  • Yes, `requirements.txt` file instead of direct enumeration in Dockerfile can be helpful. You cannot (or should not, at least) reuse packages from python 3.7 in 3.9 - at least because they may be version-incompatible and require an upgrade to use 3.9. Ideally you should know which exactly packages you need and install them in Dockerfile (probably from requirements file). – STerliakov Mar 31 '23 at 23:57
  • Even if we use a `requirements.txt` for our desired packages, this would still be a problem for launching directly the jupyter notebook from Vertex AI Workbench. I think there are still some instructions missing on that Dockerfile to be able to run it properly with Vertex (an ENTRYPOINT, maybe?). – elAndrez3000 Apr 04 '23 at 22:58
  • https://stackoverflow.com/questions/75982907/how-to-install-a-custom-container-with-latest-python-jupyterlab-version – gogasca Apr 11 '23 at 06:20
  • @gogasca Thanks! Will keep this as a future reference. – elAndrez3000 Apr 11 '23 at 23:34

1 Answers1

1

We release version m105 which includes Python 3.10. https://cloud.google.com/deep-learning-containers/docs/release-notes

gogasca
  • 9,283
  • 6
  • 80
  • 125
  • 1
    Thanks for the update @gogasca! Just for future reference, is there an optimal way to update the Python version on our own using the base DL containers? Or we'll have to wait for official releases? – elAndrez3000 Apr 04 '23 at 22:55
  • Yes, there is an official way, instead of using our base Deep Learning containers, you would need to use a different base. Example: Python 3.11 (python:3.11.1), install JupyterLab and configure JupyterLab parameters to be able to connect through our Proxy. I opened a documentation bug to track this and have an official document, in the meantime will post an answer here for reference. – gogasca Apr 06 '23 at 02:29