3
from lxml import html
import requests


url = "https://website.com/"
page = requests.get(url)
tree = html.fromstring(page.content)
page.content

-> SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

I run this script but I got this error. How can I do it?

Leo
  • 413
  • 2
  • 5
  • 11
  • [possible duplicate](https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error) check out this for ideas. – ivan7707 Oct 04 '17 at 14:05
  • 3
    It would be easier to advise if you could indicate whether "website.com" was a real URL, or just an example, and what steps you have taken to ensure that its certificate is trusted. You might need to enable some SSL tracing. – Kevin Boone Oct 04 '17 at 14:07
  • It's an internal corporate url – Leo Oct 04 '17 at 14:14
  • 2
    Possible duplicate of [urllib and "SSL: CERTIFICATE\_VERIFY\_FAILED" Error](https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error) – Mureinik Oct 04 '17 at 15:02

1 Answers1

12

Since your URL is "an internal corporate URL" (as stated in comments), I'm guessing it uses a self-signed certificate, or is issued by a self-signed CA certificate.

If that is in fact the case, you have two options:

(1) provide the path to your corporate CA (including the complete chain of intermediate certificates, if any) to requests.get() call via verify argument:

requests.get('https://website.lo', verify='/path/to/certfile')

or (2), disable client-side certificate verification altogether (but beware of all the security risks this entails, like a simple man-in-the-middle attacks, etc):

requests.get('https://website.lo', verify=False)

Fore completeness, the relevant verify parameter is described in requests.request() docs:

verify -- (optional) Either a boolean, in which case it controls whether we verify 
          the server's TLS certificate, or a string, in which case it must be a path 
          to a CA bundle to use. Defaults to True.
randomir
  • 17,989
  • 1
  • 40
  • 55
  • Where I can find my certfile? – Leo Oct 04 '17 at 16:43
  • `requests` with by default [try to use `certifi` bundle](http://docs.python-requests.org/en/master/user/advanced/#ca-certificates), but if you are asking about CA cert for your internal corporate server, the best would be to ask your network admin to provide you with their cert. – randomir Oct 04 '17 at 16:49
  • I have been struggling with a variant of OPs error for far too long today. The answer I didn't know I needed was: _requests.get('https://website.lo', verify=False)_. My Python is mad rusty, I was trying to insert the verify=False as another line item and it just wasn't doing anything. Thank you! – Cerberus136 Jan 16 '19 at 21:14
  • @Cerberus136, just be aware that setting verify to false, practically completely forgoes security provided by the SSL layer. I.e MITM attack becomes trivial, since certs can be spoofed on fly, and your client will never be aware of that. – randomir Jan 17 '19 at 01:37