So I have a python script that needs to access webpage content via 'requests'. Due to the environment this script is running in, I need to use a virtual environment. However, this results in the request failing, since it cannot find the certificate from the virtual environment.
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='...', port=443): Max retries exceeded with url: ... (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))
This issue does not occur when run outside the virtual environment, and I can see the error occur as a result of where requests it looking for certificates:
in venv (request fails):
>>> print (requests.certs.where())
/home/user/python/venv/lib64/python3.11/site-packages/certifi/cacert.pem
not in venv (request will certify successfully):
>>> print (requests.certs.where())
/etc/pki/tls/certs/ca-bundle.crt
I could specify the certification using request(url, verify=/path/to/cert) however this script needs to run on any machine, so hardcoding the path will not work.
I could ignore the verification, but this seems very dumb and bad.
Therefore I am wondering if there is a way to instruct python to use the same certification as the underlying environment rather than the venv path?
Is there some workaround where given a url, I could detect which certification will be used and provide that in my python script to the request?
Thank you
