Upgrading setuptools on OSX El Capitan

Question

I'm trying to upgrade setuptools. Well actually I'm trying to upgrade ansible but it's trying to upgrade setuptools and failing. Trying to do it myself also fails. Even trying to uninstall it fails

$ sudo -H pip install --upgrade setuptools
Collecting setuptools
  Using cached setuptools-18.4-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 1.1.6
    Uninstalling setuptools-1.1.6:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-OyTXsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

I haven't the foggiest idea what's wrong. Looking at all the files in /System/Library/Frameworks/Python.framework/Versions/2.7/ and below every single file is owned by root:wheel

How can I fix it?

I don't have a solution but I have an explaination: it's probably due to the the new [System Integrity Protection](https://support.apple.com/en-us/HT204899). pip is trying to modify things where it's not allowed to anymore, even with `sudo`. See [this question](http://stackoverflow.com/questions/33004708/osx-el-capitan-sudo-pip-install-oserror-errno-1-operation-not-permitted). I haven't taken the time to deal with this yet. The `--user` option to pip is probably what I'd look into. If you find a practical solution to "update" everything, please share! — Arthur, Oct 22 '15 at 14:55
I'm getting `ipdb` error which is linked to `setuptools`. Upgrading setuptools doesn't work. It works with `--user=python` but does not fix the ipdb error still. — R11G, Apr 13 '18 at 18:00

Kof · Accepted Answer · 2015-11-13T12:35:43.540

189

This happens due to the System Integrity Protection introduced in OS X El Capitan.

Adding --user python to the command allowed this to work.

In other words

pip install --upgrade setuptools --user python

edited Nov 13 '15 at 12:35

answered Nov 13 '15 at 07:38

Kof

23,893
9
56
81

2

`pip` (unlike `easy_install`) does not require sudo, since you're letting it use `python` user, I made sure it works. – Kof Nov 13 '15 at 12:37
1

Also works on CentOS 7 inside a Docker container. Was failing for some reason (though the exact same setup worked without this step prior to May 2016...) – geerlingguy May 02 '16 at 16:30
2

I couldn't get it to work, and I don't understand how the --user option is supposed to work. What worked for me was to disable System Integrity Protection temporarily while upgrading setuptools (in connection with installing ipython) and then re-enabling it afterwards. To do this: boot into recovery mode (cmd+R), launch the terminal and enter `csrutil disable`. Reboot, and after upgrading, reboot to recovery mode and enter `csrutil enable`. After a reboot, ipython continued to work fine for me. – Grav Jun 24 '16 at 19:21
I can execute this command successfully, but no update is made. The output looks like this: `Collecting setuptools Using cached setuptools-28.2.0-py2.py3-none-any.whl Requirement already up-to-date: python in /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload Installing collected packages: setuptools Successfully installed setuptools-1.1.6` – diidu Oct 05 '16 at 09:02
I was finally able to get things working by following the instructions given here: http://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-os-x-el-capitan-upgrade – diidu Nov 01 '16 at 07:26
This fixed worked for me when I encountered an error installing paramiko. – Ron Sims II Dec 08 '16 at 18:44
It work after adding --user python, could someone elaborate why it works? – yabchexu Nov 29 '17 at 23:46
it tells python to use the "python" user, to which required files are assigned to. maybe otherwise it tries to use currently logged user for operation. – Kof Dec 19 '17 at 07:03

Jason S · Answer 2 · 2017-08-13T02:56:56.283

Overview

The problem is conflicting Python libraries, combined with System Integrity Protection (SIP) on Mac OS which protects the system Python libraries.

The best solution in my opinion is to uninstall and re-install your own Python installation and keep that separate to the Python libraries provided by Mac OS which are protected by SIP.

I favour this to disabling SIP as I expect SIP to be a part of any future Mac OS releases and SIP is not the cause here, it just exposes the problem of conflicting Python libraries.

Details

I also had this issue trying to install ansible.

My problem started when I followed the ansible installation instructions for Mac OS, which is to install via pip and to install pip with easy_install as described at Latest Releases Via Pip

Problem is when installing pip this way, the easy_install is the Mac OS provided easy_install at /usr/bin/easy_install but it writes to the easy-install.pth file at /Library/Python/2.7/site-packages/easy-install.pth and that file references the Mac OS provided Python libraries.

Subsequently installing ansible with pip install ansible then reports that the setuptools requirement is satisfied by the Mac OS provided Python library at /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Ansible requires setuptools but doesn't specify a version. Another dependent package, paramiko, requires setuptools >= 11.3, but this doesn't appear to be checked when installing ansible via pip. When you try to run ansible though it complains that setup tools is 1.1.6 This is the version of setuptools provided with Mac OS, and is now protected by SIP, so it can't be upgraded.

Rather than disable SIP, I fixed this by uninstalling my local Python as hinted at at https://docs.python.org/2.7/using/mac.html#getting-and-installing-macpython and then downloading and installing again.

If you're concerned about the rm, you can mv instead
To uninstall I did

sudo rm -rf /Library/Python
sudo rm -rf /Applications/Python\ 2.7/
sudo rm /usr/local/bin/ansible  # executable
sudo rm /usr/local/bin/python*  # symlinks to /Library/Python/2.7
sudo rm /usr/local/bin/easy_install*
# and so on for references to /Library/Python/2.7 in /usr/local/bin

I then downloaded the 2.7.13 installer package for Mac OS X from https://www.python.org/downloads/ and installed it.

This installed a local python and pip at /Library/Frameworks/Python.framework/Versions/2.7 and symlinks in /usr/local/bin as well as prepending /Library/Frameworks/Python.framework/Versions/2.7 to my $PATH. This keeps everything separate from the Mac OS provided libraries at /System/Library/Frameworks/Python.framework and /usr/bin so that I get

which pip
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip

I then installed ansible with pip install ansible
Then when I run which ansible I get

/Library/Frameworks/Python.framework/Versions/2.7/bin/ansible

and in pip list

setuptools (28.8.0)
six (1.10.0)

and ansible now works for me because the Mac OS Python libraries are not referenced by the new install.

Note that because of the installer setting $PATH, which python is now /Library/Frameworks/Python.framework/Versions/2.7/bin/python and /usr/local/bin/python is a symlink to this.
If you want the system Python you'll have to use /usr/bin/python or change the $PATH

I K · Answer 3 · 2016-10-10T19:07:00.237

The answer is that you cannot update setuptools on OSX for the factory python that ships with the OS. The reason is that the files in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python cannot be removed or modified by the user. Not only the normal user but even if you have root permissions you will be unable to modify those files.

Sure you can use various other methods to install a newer version of setuptools but this will not override the default system package. Meaning if you go the suggested route of use the --user python flag, or you choose to install them without root into ~/Library, these will not override the system files version.

The reason you or root are unable to modify the system defaults is due to SIP permission restrictions in El Capitan+. You could disable SIP but this is generally not recommended.

Instead the only reasonable solution is to use a python virtualenv.

score 0 · Answer 4 · answered Aug 23 '23 at 00:22

0

I'm using MacbookAir (M1 2020) and this worked for me:

pip install --upgrade setuptools

answered Aug 23 '23 at 00:22

Cyebukayire

795
7
14

Upgrading setuptools on OSX El Capitan

4 Answers4

Overview

Details

Linked