10

I'm trying to make an HTTPS request using Curl through a squid proxy. I know that the squid proxy works, since I have set it up for my browser and it works fine. I have tried using just about every answer from here and scoured several other websites, but nothing turns up.

Some sample searches and results: 1) using basic auth inline: curl -x https://user:pass@host:port https://www.google.com -v

Result: Establish HTTP proxy tunnel to www.google.com:443 Proxy auth using Basic with user 'username' CONNECT www.google.com:443 HTTP/1.1 Host: www.google.com:443 Proxy-Authorization: Basic abaskldfja1fiopweifj= User-Agent: curl/7.47.0 Proxy-Connection: Keep-Alive Recv failure: Connection reset by peer Received HTTP code 0 from proxy after CONNECT Closing connection 0 curl: (56) Recv failure: Connection reset by peer

2) using env vars (https_proxy and http_proxy): same result

3) Putting credentials in argument: curl -x https://host:port https://www.google.com -v --proxy-user user:pass: same result

Any guesses on what I could be doing wrong?

RyanQuey
  • 705
  • 1
  • 9
  • 29
  • hey @RyanQuey have you managed to work this out? – Rico Apr 10 '18 at 15:34
  • @RicoW I never figured this issue out. Would be nice though for troubleshooting! Ended up solving my particular problem by a workaround: Just used the "phone home" command in the squid's cloud init file to let us know when it was ready. – RyanQuey Apr 10 '18 at 16:39

1 Answers1

1

You haven't provided enough information to determine why you have your problem.

For example:

How is your squid https proxy configured? Is the proxy operating in splice or bump mode?

Are you absolutely sure your proxy is working?

Did you try connecting to any other sites via http or https?

Are their any other proxy authentication options set? Restrictions on IP addresses that can use the proxy? What authentication option did you configure? Did it work without authentication enabled?

For what it is worth, I needed to do the same for my own reasons. I first configured the proxy in "splice all" mode and here was the result showing headers only:

$ curl -x 10.10.1.1:3128 -I https://www.google.com/
HTTP/1.1 200 Connection established

HTTP/2 200
content-type: text/html; charset=ISO-8859-1
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
date: Mon, 04 Apr 2022 12:14:56 GMT
server: gws
x-xss-protection: 0
x-frame-options: SAMEORIGIN
expires: Mon, 04 Apr 2022 12:14:56 GMT
cache-control: private
[snip]

Next, I configured the proxy in "splice whitelist, bump otherwise" mode and tried again:

# curl -x 10.10.1.1:3128 -I https://www.google.com/
HTTP/1.1 200 Connection established

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This was expected.

Using the -k option allows it to work (ignore cert errors):

# curl -x 10.10.1.1:3128 -I https://www.google.com/ -k
HTTP/1.1 200 Connection established

HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Mon, 04 Apr 2022 12:34:21 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Mon, 04 Apr 2022 12:34:21 GMT
Cache-Control: private
[snip]

or using the cert defined in the https proxy settings:

$ curl -x 10.10.1.1:3128 --cacert ~/test/my-MITM.crt -I https://www.google.com/
HTTP/1.1 200 Connection established

HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Mon, 04 Apr 2022 12:35:06 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Mon, 04 Apr 2022 12:35:06 GMT
Cache-Control: private
[snip]

Next, I enabled authentication (still in bump mode, ignoring cert errors) and it didn't like that, as expected

$ curl -x 10.10.1.1:3128 -k -I https://www.google.com/
HTTP/1.1 407 Proxy Authentication Required
Server: squid/4.15
Mime-Version: 1.0
Date: Mon, 04 Apr 2022 12:40:46 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3532
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
Proxy-Authenticate: Basic realm="Please enter your credentials to access the proxy"
X-Cache: MISS from pfsense
X-Cache-Lookup: NONE from pfsense:3128
Via: 1.1 pfsense (squid/4.15)
Connection: keep-alive

curl: (56) Received HTTP code 407 from proxy after CONNECT

So let's try with authentication:

$ curl -x hello:world@10.10.1.1:3128 -k -I https://www.google.com/
HTTP/1.1 200 Connection established

HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Mon, 04 Apr 2022 12:43:09 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Mon, 04 Apr 2022 12:43:09 GMT
Cache-Control: private
[snip]

And we're good.

Since your error didn't match anything I'd seen, I thought I'd try one more exercise. Rather than not specify the protocol as part of the proxy server definition, I added it in:

$ curl -x https://hello:world@10.10.1.1:3128 -k -I https://www.google.com/
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

ah. interesting. let's get some detail:

$ curl -x https://hello:world@10.10.1.1:3128 -k -I https://www.google.com/ -v
*   Trying 10.10.1.1...
* TCP_NODELAY set
* Connected to 10.10.1.1 (10.10.1.1) port 3128 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
* Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

That looks closer to your error.

Final test, specify http rather than https for the proxy server

$ curl -x http://hello:world@10.10.1.1:3128 -k -I https://www.google.com/
HTTP/1.1 200 Connection established

HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Mon, 04 Apr 2022 12:51:27 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Mon, 04 Apr 2022 12:51:27 GMT
Cache-Control: private
[snip]

And there you have it, and that's good enough for me.

I'm guessing here, but it looks like if you specify a protocol as part of the proxy string, it will try to use that protocol to communicate with the proxy server. So using http://, or not specifying it as I did original works just fine, but as soon as I said https: ... ☠️☠️☠️

I hope that helps for anyone with interest in this little bit of trivia.

pdwalker
  • 823
  • 7
  • 8
  • Thanks for this - I have since moved on and probably won't ever come back to this, so can't verify your solution. But if enough other people upvote then I can accept it. – RyanQuey Apr 04 '22 at 15:58
  • No worries. I was just documenting it here in case someone else needed a solution to the problem. I've since learned a lot more about squid and configuring clients to use the proxies. Pretty nifty stuff actually. – pdwalker May 17 '22 at 06:32