How to upgrade python on mac if multiple versions (including upgraded version) are already installed?

Question

I upgraded pip using the command pip3 install --upgrade pip; then, the following was printed in the terminal window:

DEPRECATION: Python 3.5 reached the end of its life on September 13th,
2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021.
pip 21.0 will remove support for this functionality.

Searching online about the right way to upgrade python on a mac lead me to two sources (1) and (2). The second source mentions modifying the .profile and bash script; these are both things I am having trouble understanding, despite some help from similar questions on these stack exchange forums like this and that. But, using suggestions from the first source, I think I've identified a potential future problem (for which I read virtual venv may be a good fix). Specifically,

$ which python 
/usr/bin/python

$ which python3
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3

$ brew info python
python@3.8: stable 3.8.5 (bottled)
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/python@3.8/3.8.5 (4,331 files, 66.8MB)
  Poured from bottle on 2020-09-23 at 04:16:21
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/python@3.8.rb
License: Python-2.0
==> Dependencies
Build: pkg-config ✔
Required: gdbm ✔, openssl@1.1 ✔, readline ✔, sqlite ✔, xz ✔
==> Caveats
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python@3.8/libexec/bin

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.8/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Analytics
install: 435,972 (30 days), 1,606,776 (90 days), 3,198,770 (365 days)
install-on-request: 158,335 (30 days), 466,841 (90 days), 519,965 (365 days)
build-error: 0 (30 days)

I think this means I have python 2, python 3.5, and python 3.8 installed on my mac. I tried upgrading my python version using the multiple versions of following command (ie, python, python3, python3.8, etc):

$ pip3 install --upgrade python
ERROR: Could not find a version that satisfies the requirement python (from versions: none)
ERROR: No matching distribution found for python

I have installed python 3.8, but I'm apparently running 3.5; how do I upgrade my version of python? And how do I "clean this mess up"? In case it's relevant, I'm running macOS High Sierra 10.13.6, and the only text-editor I like/use is Atom (please no suggestions regarding vim).

EDIT #1:

After following the steps from this post, the following was opened in TextEdit.

# Setting PATH for Python 3.5
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}"
export PATH

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

I notice that it says Python 3.5 at the top. Do I over-write the given path PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}" with PATH=/usr/local/bin? I would like to run python 3.8; will this do what I want it to?

EDIT #2:

As suggested in the edited answer, running grep PATH $HOME/.* in the terminal outputs the following in the terminal window:

grep: /Users/username/.bash_sessions: Is a directory
grep: /Users/username/.cache: Is a directory
grep: /Users/username/.config: Is a directory
grep: /Users/username/.cups: Is a directory
grep: /Users/username/.git: Is a directory
grep: /Users/username/.idlerc: Is a directory
grep: /Users/username/.lightkurve-cache: Is a directory
grep: /Users/username/.local: Is a directory
grep: /Users/username/.matplotlib: Is a directory
grep: /Users/username/.npm: Is a directory
grep: /Users/username/.nvm: Is a directory
grep: /Users/username/.oracle_jre_usage: Is a directory
grep: /Users/username/.ssh: Is a directory
grep: /Users/username/.subversion: Is a directory
grep: /Users/username/.vnc: Is a directory

So, I search for the directory containing python 3.5. Frameworks is not listed if I ls while in /Users/username/Library/, so I search elsewhere. But this worries me because I saw PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}" (from edit #1).

$ pwd
/Users/username

$ cd .local/
$ ls 
share

$ cd share
$ ls
virtualenvs

$ cd virtualenvs
$ ls
PyCodes-_a3khG54    

$ cd PyCodes-_a3khG54
$ ls
bin include lib src

The directory src is empty; include contains python3.5m and lib contains python3.5.

$ cd bin
$ ls

activate        f2py            python
activate.csh        f2py3           python-config
activate.fish       f2py3.5         python3
activate_this.py    pip         python3.5
easy_install        pip3            wheel
easy_install-3.5    pip3.5

Shouldn't there be a file here that I can edit?

Answer 1

The output you show from:

brew info python

tells you the answer!

If I paraphrase, it says:

If you want to run Python 3.8 and pip by typing python3 and pip3, you need to put /usr/local/bin first on your PATH. If, instead, you want to run them by typing python or pip, i.e. without a version, you need to put /usr/local/opt/python@3.8/libexec/bin first on your PATH.

It's not that straightforward to say where to change your PATH because there are many possibilities as to where it is set. Your best bet is probably to look in all "dotfiles" in your HOME directory, so:

grep PATH $HOME/.*

Then you need to open the most likely looking file, and add just after the last place it is set, something like:

export PATH=/usr/local/bin:$PATH

To actually activate it, the simplest is probably to start a new Terminal and run:

echo $PATH

If it looks correct, close the Terminal with the old PATH else you'll confuse yourself. If it looks incorrect, or causes an error, immediately close the new Terminal window and retry the edit in the original Terminal.

---

If you want to know what you are actually running when you type python, and pip, run:

type python
type pip

---

If you want to know what you are actually running when you type python3, and pip3, run:

type python3
type pip3

---

You can interpret the outputs of the last few type commands like this:

• if they contain the word local near the start, probably /usr/local/XXX, they are likely supplied by homebrew or something you have compiled or installed yourself,

• if they start with /Library or contain the word Frameworks, they are probably supplied by Apple and fairly recent,

• if they start with /usr/bin but without the word local, they are probably older Apple-supplied binaries,

• if it says your command is "aliased", that means you have a command like alias python="something crazy" somewhere in your "dotfiles" (see above). Personally, I would remove that as it just adds another layer of indirection and confusion... YMMV on this one.

Never delete or alter anything supplied by Apple - else future macOS upgrades may fail.

---

So, hopefully we have established that you need to either use python and pip, or use python3 and pip3 and that they should match, i.e. that when you install a package you are installing it in a place usable by the Python interpreter you use. So, an example of a "matching" installation looks like this:

type python3
python3 is /usr/local/bin/python3

python3 -V
Python 3.8.5

pip3 -V
pip 20.1.1 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)

type pip3
pip3 is hashed (/usr/local/bin/pip3)

Notice how both my binaries contain local in their names, so they both come from homebrew and how the Python version matches the version in the pip3 -V command so I can be sure I am installing in the correct place for my Python interpreter.

---

Why is it hard to say where to edit your PATH? The answer is that it depends on many things. It depends on your shell, so you might see the advice to run echo $SHELL, but that only tells you your default shell, not your current shell, which you can find with ps -o comm= $$. But your login shell could be bash and you may have execed fish in your bash profile. Or you may use iTerm2 or Terminal or xterm and you may have changed your shell in the "Preferences" for that terminal program.