You can use the method on top and install using apt... It's the best way to do it,
sudo apt update
sudo apt install software-properties-common
Followed by adding the repository,
sudo add-apt-repository ppa:deadsnakes/ppa
Accept the changes, then
sudo apt install python3.9
Now, whenever you want to use python3.9 you have to invoke python3.9 instead of python3.
If you are using cloudera, cdsw, you cannot get sudo rights, you need to follow this guide to install packages,
https://docs.cloudera.com/documentation/data-science-workbench/1-8-x/topics/cdsw_extensible_engines.html
Follow the guide above, then change the docker file like this
# Dockerfile
FROM docker.repository.cloudera.com/cdsw/engine:8
RUN rm /etc/apt/sources.list.d/*
RUN apt-get update
RUN apt install software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt install python3.9 python3-pip \
&& rm /etc/apt/sources.list.d/*
RUN pip install pandas numpy
Then follow the rest of the guide for steps 2-4, you should be able to get your desired outcome.