0

I tried to crawl https://foodtracer.taipei.gov.tw/Front/Chain/Product?id=12411160 with

> import requests
> `url = 'https://foodtracer.taipei.gov.tw/Front/Chain/Product?id=12411160'
>
> r = requests.post(url)
> 
> print(r.status_code) # 200
> 
> if r.status_code == requests.codes.ok: # OK!   print("OK!")`

But this occurred:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='foodtracer.taipei.gov.tw', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

requests.exceptions.SSLError: HTTPSConnectionPool(host='foodtracer.taipei.gov.tw', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

I tried/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -m pip install --upgrade pip but the error still occurred.
Is there any other way for me to crawl this website?

WinnieXi
  • 13
  • 1
  • 4

1 Answers1

2

You need to add verify=False in your request:

r = requests.post(url, verify=False)

This will generate a warning, though you can choose to suppress it.

Jarvis
  • 8,494
  • 3
  • 27
  • 58
  • 1
    Thanks! But I still don't know why. Could someone explain? Is there any other way for me to do this? Cause this seems unsafe. – WinnieXi Dec 23 '20 at 13:13
  • If the main issue is resolved, please mark the answer as accepted. SSL issue must be due to the fact that the website's SSL certificate has expired and hence can't be verified by any CA. @WinnieXi Don't think you can really do something here, it's up-to the site's administrators to provide a valid SSL certificate for their website. – Jarvis Dec 23 '20 at 13:14
  • You can check the website’s certificate in your browser - in the address bar it will show a lock with a line or X through it - you can click this and examine the certificate details. – DisappointedByUnaccountableMod Dec 23 '20 at 13:39
  • what is the permenant solution for this? Verify false is a temporary soln. my browser not showing the certificate is invalid. but some requests this comes as response – Sam Jun 22 '22 at 11:26