5

I am facing the SSLError, when trying to scrape websites.

import requests
url = 'https://www.amazon.com/'
page = requests.get(url)
content=page.content
print(content)

Output:~\Anaconda3\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 429 except (_SSLError, _HTTPError) as e: 430 if isinstance(e, _SSLError): --> 431 raise SSLError(e, request=request) 432 elif isinstance(e, ReadTimeoutError): 433 raise ReadTimeout(e, request=request)

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)

there is a workaround that works:

import requests
url = 'https://www.amazon.com/'
page = requests.get(url=url, verify =False)
content=page.content
print(content)

But I would love to settle the issue with my certificates!

I have updated everything including requests, reinstalled my Anaconda3, checked my certificates with https://www.ssllabs.com - they are ok.

System inf: Windows 10, pip version 20.0.2 ,anaconda3,python 3.7

ANY IDEA on WHAT does THAT DAMN 1076 ERROR particularly refer to and HOW TO FIX it?

Thanks a ton in advance

Victoria
  • 247
  • 4
  • 8
  • I think verify False skips ssl, and connects via http. That doesn't seem like a workaround but avoids connecting to HTTPS – getflag Apr 20 '20 at 08:49
  • yes that is it the https connection fails - can`t see why, checked my verify- its fine – Victoria Apr 20 '20 at 20:30
  • I actually looked it up online and found a few links pointing me to github and they have had some bug activity on it. I feel this is a known thing. – getflag Apr 20 '20 at 23:28
  • Could be - anyway I decided to try installing Anaconda 4 - that might solve it..... – Victoria Apr 21 '20 at 09:14

1 Answers1

2

It was the 'wonderful' Zscaler installed by global IT as the company policy now requires, I got the certificate from the IT department - strugling to install it to Anaconda3 now posting this question here

Victoria
  • 247
  • 4
  • 8
  • after your Zscaler root cert is installed in the Windows trust root store, just install either pip-system-certs or python-certifi-win32 from either pypi or conda-forge – Mark Mikofski Jan 23 '23 at 06:57