Pip broken after upgrade. Cannot unsinstall or upgrade Pip or Python

Question

I recently had tried to run a pip command and noticed my pip version was considerably out of date so upgraded it. This seems to have completely broken my pip install and I cannot figure out how to reset it.

$ pip3 --version
pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)
You are using pip version 8.1.1, however version 22.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ pip3 install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/f3/77/23152f90de45957b59591c34dcb39b78194eb67d088d4f8799e9aa9726c4/pip-22.1-py3-none-any.whl (2.1MB)
    100% |████████████████████████████████| 2.2MB 505kB/s 
Installing collected packages: pip
Successfully installed pip-8.1.1
You are using pip version 8.1.1, however version 22.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ pip3 --version
Traceback (most recent call last):
  File "/usr/bin/pip3", line 11, in <module>
    sys.exit(main())
  File "/home/myname/.local/lib/python3.5/site-packages/pip/__init__.py", line 11, in main
    from pip._internal.utils.entrypoints import _wrapper
  File "/home/myname/.local/lib/python3.5/site-packages/pip/_internal/utils/entrypoints.py", line 12
    f"pip{sys.version_info.major}",
                                 ^
SyntaxError: invalid syntax

This seems to come from this: https://github.com/pypa/pip/issues/9526

I can't seem to uninstall pip:

$ python3 -m pip uninstall pip
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/myname/.local/lib/python3.5/site-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/home/myname/.local/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 57
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

I'm on python3.5 and can't seem to upgrade that either:

$ sudo apt-get install python3.6
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package python3.6
E: Couldn't find any package by glob 'python3.6'
E: Couldn't find any package by regex 'python3.6'

I'm found a number of stackoverflow answers suggesting to use alternative repositories but they're not working either.

$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt-get update
$ sudo apt-get install python3.6
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package python3.6
E: Couldn't find any package by glob 'python3.6'
E: Couldn't find any package by regex 'python3.6'

I've tried 3.6, 3.7, 3.8, 3.9 and 3.10 with the same results.

I've also found some stack overflow answers suggesting to edit /usr/bin/pip3

Originally it looks like this:

from pip import main
if __name__ == '__main__':
    sys.exit(main())

I've tried this:

from pip._internal import __main__
if __name__ == '__main__':
    sys.exit(main())

and this: I've tried this:

from pip._internal import __main__
if __name__ == '__main__':
     sys.exit(__main__._main())

But neither have resolved the issue.

To be sure i'm not messed up with a local version i've tried all of the above using /usr/bin/pip3 --version rather than just pip3 --version with the same results.

Now i'm at a bit of a loss how to get a working version of Python and Pip.

---

Suggestion from question:

$ sudo python3.5 -m easy_install pip
Searching for pip
Best match: pip 22.1
Adding pip 22.1 to easy-install.pth file
Installing pip3 script to /usr/local/bin
Installing pip3.10 script to /usr/local/bin
Installing pip script to /usr/local/bin

Using /home/myname/.local/lib/python3.5/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

$ ls -lh /usr/local/bin/*pip*
-rwxr-xr-x 1 root root 361 May 12 19:49 /usr/local/bin/pip
-rwxr-xr-x 1 root root 363 May 12 19:49 /usr/local/bin/pip3
-rwxr-xr-x 1 root root 369 May 12 19:49 /usr/local/bin/pip3.10

All 3 return the same thing:

$ /usr/local/bin/pip --version
$ /usr/local/bin/pip3 --version
$ /usr/local/bin/pip3.10 --version
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    load_entry_point('pip==22.1', 'console_scripts', 'pip')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/myname/.local/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 57
    sys.stderr.write(f"ERROR: {exc}")
                                   ^

---

I've now installed Python3.6 using "method two" from here: https://askubuntu.com/questions/1152640/apt-install-doesnt-install-python3-6 i.e. manual install

I've then upgraded pip using:

sudo python3.6 -m pip install pip --upgrade

Now pip --version and pip3 --version return an error but directly accessing the bin works:

$ /usr/local/bin/pip --version
pip 21.3.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

The aliases are currently pointing to:

$ which pip
/home/sortitoutsi/.local/bin/pip
$ which pip3
/home/sortitoutsi/.local/bin/pip3

---

It seems that other things are now broken, such as aws.

$ aws --help
Traceback (most recent call last):
  File "/usr/local/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 17, in <module>
    import botocore.session
  File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 29, in <module>
    import botocore.configloader
  File "/usr/local/lib/python2.7/dist-packages/botocore/configloader.py", line 19, in <module>
    from botocore.compat import six
  File "/usr/local/lib/python2.7/dist-packages/botocore/compat.py", line 26, in <module>
    from dateutil.tz import tzlocal
  File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/__init__.py", line 1, in <module>
    from .tz import *
  File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/tz.py", line 16, in <module>
    from six import string_types, PY3
ImportError: No module named six

---

Simply reinstalling aws sudo python -m pip install awscli seemed to have fixed the above. I'm guessing a sudo apt-get update at some point forced an upgrade of awscli after finding an updated Python installation.

Answer 1

Do not try and install Python versions and packages (from another ppa) that are not supported by your OS. Your system may break.

Your attempted Pip installation is, luckily, in a local directory: $HOME/.local/. Check if there is anything else installed in $HOME/.local/bin/ and $HOME/.local/lib/python3.5/site-packages/. If nothing (or nothing that seems important), rename that directory:

mv ~/.local ~/.local_aside

Now, you should have the system Pip back again.

You may lose some other packages that you installed earlier (which would now be in ~/.local_aside, but possibly, you can reinstall those. Either with a system packages (e.g., python3-numpy), or again with pip3 install numpy and the like. Though pip may have trouble finding a correct NumPy version, since Python 3.5 is not supported anymore (the system package should be fine).

---

If you want to use a more recent version of Python, install a local (user-only) one. My preferred way of doing that is to use pyenv, which requires some set up (installing the necessary build packages). If you use pyenv, carefully read the installation instructions for your case. Once you get it going, you can quickly install any version of Python, and between minor versions, they won't even get in the way (that is, python3.9 -m pip install numpy will be a separate installation from python3.10 -m pip install numpy, and both can happily live next to each other if so wanted).

But don't use sudo and install it as a root: your system will get confused and may break.

---

Another alternative is Conda, in the form (my preference) of miniconda3. Conda can and will also install C library dependencies, and actively tries to resolve version conflicts between Python packages (in case you have to install a lot of packages). Again, browse the manual to get going, but it's relatively straightforward to use, and doesn't get in the way of your OS.