SSL Request Failed - Python OSX

Question

When accessing an authenticated webpage with cookies and am getting an SSL Error. This seems to be a mac-specific issue that many solve by installing the python certificates (as I tried in the terminal session pictured below), but an exception throws (Permission Denied) when I attempt at installing.

How should I fix this error?

Code:

...
import urllib.request

opener = urllib.request.build_opener()
print("Cookies: ", headers["Cookie"])
opener.addheaders.append(('Cookie', headers["Cookie"]))
f = opener.open("https://home.nest.com/home/"+url_addtl_data)
print(f)

Error Message:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Ryan/Desktop/Dev/Projects/Py/Database_intro/venv/main.py", line 97, in <module>
    f = opener.open("https://home.nest.com/home/"+url_addtl_data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

Solution attempted:

Last login: Sat May 11 03:44:45 on ttys000
Ryan-Cocuzzos-Laptop:~ Ryan$ /Applications/Python\ 3.6/Install\ Certificates.command ; exit;
 -- pip install --upgrade certifi
Collecting certifi
  Using cached https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl
Installing collected packages: certifi
Exception:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi-2019.3.9.dist-info'
You are using pip version 9.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Traceback (most recent call last):
  File "<stdin>", line 44, in <module>
  File "<stdin>", line 25, in main
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6', '-E', '-s', '-m', 'pip', 'install', '--upgrade', 'certifi']' returned non-zero exit status 2.
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Deleting expired sessions...30 completed.

[Process completed]

EDIT 5/19

Additional Terminal Sessions (from suggested solutions):

Ryan-Cocuzzos-Laptop:~ Ryan$ sudo easy_install pip
Password:
Searching for pip
Best match: pip 19.1.1
Adding pip 19.1.1 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip3.7 script to /usr/local/bin
Installing pip3 script to /usr/local/bin

Using /usr/local/lib/python2.7/site-packages
Processing dependencies for pip
Finished processing dependencies for pip
Ryan-Cocuzzos-Laptop:~ Ryan$ brew install python
Updating Homebrew...
Warning: python 3.7.3 is already installed, it's just not linked
You can use `brew link python` to link this version.
Ryan-Cocuzzos-Laptop:~ Ryan$ brew link python
Linking /usr/local/Cellar/python/3.7.3... 
Error: Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
  rm '/usr/local/bin/2to3'

To force the link and overwrite all conflicting files:
  brew link --overwrite python

To list all files that would be deleted:
  brew link --overwrite --dry-run python
Ryan-Cocuzzos-Laptop:~ Ryan$ rm '/usr/local/bin/2to3'
Ryan-Cocuzzos-Laptop:~ Ryan$ brew link --overwrite python
Linking /usr/local/Cellar/python/3.7.3... 24 symlinks created
Ryan-Cocuzzos-Laptop:~ Ryan$ python -c "import ssl;print(ssl.OPENSSL_VERSION)"
OpenSSL 1.0.2r  26 Feb 2019
Ryan-Cocuzzos-Laptop:~ Ryan$

EDIT 5/21

Attempted to include SSL in the project and it threw an error.. this appears to be relevant, but, when checking (and as is shown above), you can notice that python is definitely using OpenSSL v. 1.0.2r.

What does `python -c "import ssl;print(ssl.OPENSSL_VERSION)"` output when run from your shell? You will get around the 2nd error by installing (upgrading) *pip* manually: https://bootstrap.pypa.io/get-pip.py. — CristiFati, May 19 '19 at 10:19
@CristiFati Attached the session of both that statement and upgrading pip (and python actually), but neither made a difference. The error still exists. Worth noting that I did NOT upgrade pip manually, was unsure as to how with the file you linked. — Ryan Cocuzzo, May 19 '19 at 19:07
Now that you installed python multiple times, you need to make sure you are using the one you intend to use when you call `python` in the command line. Using the full path can maybe help to solve your problem or help others help you solve it. See also https://stackoverflow.com/a/51970236/6018688 — fabianegli, May 23 '19 at 07:57
In your update 5/21 you try to run python 2 code with python 3. Try to find out why this happens and if my previous comment can help resolve this issue. — fabianegli, May 23 '19 at 08:08

fabianegli · Accepted Answer · 2019-05-23T07:49:23.360

1

There should be no need to either re-install Python nor pip.

From https://stackoverflow.com/a/42098127/6018688

Just browse to Applications/Python 3.6 and double-click Install Certificates.command

The location for the Install Certificates.command might differ for python installations by brew.

There are other Questions/Answers around certificates and openssl in this environment here, here and here

Update: The permission denied error clearly points to the lack of permissions for the install. This answer and its comments suggesting the use of sudo and sudo with the -H flag may help you solve the issue: https://stackoverflow.com/a/49953581/6018688

edited May 23 '19 at 07:49

answered May 21 '19 at 19:57

fabianegli

2,056
1
18
35

I tried to flag the question as duplicate, but it seems that can't be done with questions having an open bounty. – fabianegli May 21 '19 at 19:58
The accepted and subsequent responses on the questions you linked are included in the Attempted Solutions above, minus one caveat which was interesting but still not helpful. (Attached at bottom of question). Also I asked and put a bounty on the question as I considered it fairly urgent and no other questions had working answers. – Ryan Cocuzzo May 21 '19 at 23:30
1

Did you try using `sudo` to install the certificates? – fabianegli May 23 '19 at 07:28
I have the same problem and need help. When I click the Install Certificate file, it shows an error ```ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/certifi' Consider using the `--user` option or check the permissions. ``` I don't know where to add the --user? Because it's just click the file, not typing it. – Megan Jul 31 '22 at 17:41
@Megan You can either try the command, as suggested, with the `--user` flag (enter `/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/certifi --user`) or with sudo (`sudo /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/certifi`). If a command is prepended with sudo it gets permissions to change otherwise protected files. – fabianegli Aug 04 '22 at 16:59
@fabianegli That's a wise idea! Thanks – Megan Aug 05 '22 at 12:11

score 0 · Answer 2 · answered May 19 '20 at 10:50

MacOS Catalina (10.15.4). IDE: PyCharm

I've had a similar error. My first shot was to:

brew update && brew upgrade

brew uninstall --ignore-dependencies openssl; brew install https://github.com/tebelorg/Tump/releases/download/v1.0.0/openssl.rb

Keep in mind, that I had to use --ignore-dependencies flag, because other packages installed that depend on OpenSSL.

Then I tried to reinstall Python using:

brew reinstall python

In the end the resolution was to delete venv, create it once again, and install all dependencies.

SSL Request Failed - Python OSX

2 Answers2