curl: (60) SSL certificate problem: when uploading behind proxy

Question

I need to do curl uploading behind company proxy. and I've getting the following two type of problems depending on the site that I try,

• curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
• curl: (60) SSL certificate problem: unable to get local issuer certificate

Here are the details:

Case 1:

. . . 
< HTTP/1.1 200 Connection established
< Proxy-agent: CCProxy
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Case 2:

$ curl -vX POST -d "userId=5&title=Hello World&body=Post body." https://jsonplaceholder.typicode.com/posts
Note: Unnecessary use of -X or --request, POST is already inferred.
* Uses proxy env variable https_proxy == 'http://10.xx.xx.xx:808/'
*   Trying 10.xx.xx.xx:808...
* TCP_NODELAY set
* Connected to 10.xx.xx.xx port 808 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to jsonplaceholder.typicode.com:443
> CONNECT jsonplaceholder.typicode.com:443 HTTP/1.1
> Host: jsonplaceholder.typicode.com:443
> User-Agent: curl/7.68.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< Proxy-agent: CCProxy
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The problem is not the above CCProxy, but our company is using the Zscaler transparent proxy which is intercepting SSL requests with its own certificate.

Is there any way to fix it pls?

$ curl --version
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.8.0 nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08

$ lsb_release -a 
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux bullseye/sid
Release:        testing
Codename:       bullseye

Answer 1

Step 1 in both options will extract the Zscaler certificates.

OPTION 1 Direct curl

1. Download the certificates (all certificates are included in a single file)
2. Execute the curl command passing the certificateS you want to use.

# 1
openssl s_client -showcerts \
-connect jsonplaceholder.typicode.com:443 </dev/null 2>/dev/null \
| sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p'  > typicode.crt
# 2
curl --cacert typicode.crt -v \
-d "userId=5&title=Hello World&body=Post body." \
 https://jsonplaceholder.typicode.com/posts

OPTION 2 (installer script)

In case the curl command is executed by an installer you don't have control, then, update your certificates:

1. Extract the certificates from server (use the FQDN or IP and PORT, i.e: jsonplaceholder.typicode.com:443)
2. Move the XXX.crt certificate to your certificates directory
3. Update certificates
4. Execute installation script

# 1
openssl s_client -showcerts \
-connect jsonplaceholder.typicode.com:443 </dev/null 2>/dev/null \
| sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p'  > typicode.crt
# 2
sudo mv typicode.crt /usr/local/share/ca-certificates/
# 3
sudo update-ca-certificates
# 4 execute your installer script

Bonus

In case you need/want to get the Zscaler certificates only, get the IP from: https://ip.zscaler.com

openssl s_client -showcerts -servername server -connect 165.225.216.33:443  </dev/null 2>/dev/null | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | grep -m1 -B-1 -- '-----END CERTIFICATE-----'  > zscaler.crt

UPDATED (11/19/21):

• Adding option 1, when is a direct curl and no need of install the certificates.
• Optimized the command for extracting the certificates (creating the file)
• Bonus: Getting the Zscaler IP

Tested on Ubuntu 20 and 18 behind Zscaler proxy.

Without certificate

With certificate

References:

• How to install certificates for command line
• unable to connect to server: x509: certificate signed by unknown authority

Answer 2

The answer is to "add that proxy's certificate to the CA bundle", thanks to Daniel Stenberg's answer. Then I guess I am suppose to fill in the rest. So here it is my attempt solving the remaining of the problems/questions --

• Q: What is the easiest way to get that Zscaler certificate?
  A: From here:

Go to Policy > SSL Inspection. In the Intermediate Root Certificate Authority for SSL Interception section, click Download Zscaler Root Certificate. Navigate to the ZscalerRootCerts. zip file and unzip it.

• Q: How to add that certificate to the CA bundle?
  A: See How to install company proxy certificate:

• You can use curl --cacert <CA certificate> to supply your company CA cert.
• Or you can add your company CA cert to /etc/pki/tls/certs/ and run make there to make it available system-wide.

Answer 3

This error (SSL certificate problem) means that the CA store that curl uses to verify the server's peer did not contain the cert and therefore the server couldn't be verified.

If want curl to work with a transparent proxy that terminates TLS you must add that proxy's certificate to the CA bundle or completely ignore the certificate check (which I recommend against).

A transparent proxy for TLS will of course make the connection completely unreliable and have broken security properties.