1

How in python using requests package to make an https request through a proxy with disabling certificate verification (analogous to the --proxy-insecure key in curl)? My code:

import requests
requests.get('https://ip.quotaguard.com', proxies={'https': 'login:password@eu-west-shield-01.quotaguard.com:9294'}, verify=False)

verify=False is also used, but this is not relevant to the question, since this is an analogue of the -k switch in curl, it disables verification on ip.quotaguard.com, and on proxy eu-west-shield-01.quotaguard.com:9294 does not.

The code throws the error ValueError: check_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED.

Through curl, the request successfully returns the result, here is the request itself:

curl -x "https://login:password@eu-west-shield-01.quotaguard.com:9294" -L "https://ip.quotaguard.com" --proxy-insecure -k

If there is such a possibility, but not in requests, please recommend another library.

Vladimir Pankov
  • 377
  • 3
  • 8

1 Answers1

1

Vladimir, I saw you're using QuotaGuard to solve this issue. I'm sorry for the delay in catching this question to get you an answer. We're just spinning up our support on SO, so again, apologize for the delay.

For your question, the first sentence in the following link is applicable: Requests is a great HTTP library for Python. It allows you to specify an authenticated proxy on a per request basis so you can pick and choose when to route through your static IP.

https://devcenter.heroku.com/articles/quotaguardshield#https-proxy-python-django

The requests library uses HTTP proxy only, not HTTPS proxy. So when listing HTTPS in the options, you are using an HTTP proxy for HTTPS requests, not an HTTPS proxy for HTTP(S) requests. Therefore, in your case, QuotaGuard Static would work, not QuotaGuard Shield. (Your connection string looks like you're on Shield)

It might help to check out the differences between QuotaGuard Static and QuotaGuard Shield, as it explains a bit why that's the case:

https://www.quotaguard.com/why-is-quotaguard-shield-more-secure-than-quotaguard-static/

I hope that helps, let me know here or reach out to us on Support if you need more assistance with the setup.

QuotaGuard
  • 50
  • 5
  • Thanks for the reply, I'm glad you showed up on SO. I tried to solve the problem locally by modifying the source code for requests / urllib3, which seems to work, but I'm not sure yet if the same can be done on heroku. I'll try a little later and your solution. I use "QuotaGuard Shield Static IP's" plan "Micro", before that I used "QuotaGuard Static IP's" which only worked with http, but I needed to switch to https interaction with the external system (incoming and outgoing SSL requests with static addresses), and I couldn't do it on "QuotaGuard Static IP's". – Vladimir Pankov Apr 22 '21 at 12:01
  • Perhaps you can tell me how to make https interaction between systems using http-proxy – Vladimir Pankov Apr 22 '21 at 12:04
  • Sorry for the delay, but there's a reason for it... – QuotaGuard Apr 23 '21 at 15:27
  • We did some testing over the last few days and found that requests DOES now support HTTPS proxies. So we had to update some documentation on how to use it on Heroku and I wanted to circle back to you too. My engineers were looking at this thread and their question now is, “why are they trying to use the insecure method. They should not need that to connect to our proxy, as its already secure”. Not sure if that helps or opens up a new can of questions. Go ahead and post them here and we’ll get this all squared away for you. – QuotaGuard Apr 23 '21 at 15:36
  • "why are they trying to use the insecure method" -- because I don't know yet how to specify the certificate for the proxy in `python requests`, it seemed to me easier to disable the certificate – Vladimir Pankov Apr 26 '21 at 20:59
  • The proxy uses HTTPS by default. Therefore, the requests library will perform standard HTTPS connection between the library (requests) and the proxy (QG Shield). Are you trying to connect through the HTTPS proxy to an HTTP endpoint? – QuotaGuard Apr 29 '21 at 15:50
  • No, the endpoint is https, but it has an invalid certificate (I don't know if this matters) – Vladimir Pankov May 11 '21 at 09:02