How to do a https request with bad certificate?

Question

Say I want to get https://golang.org programatically. Currently golang.org (ssl) has a bad certificate which is issued to *.appspot.com So when I run this:

package main

import (
    "log"
    "net/http"
)

func main() {
    _, err := http.Get("https://golang.org/")
    if err != nil {
        log.Fatal(err)
    }
}

I get (as I expected)

Get https://golang.org/: certificate is valid for *.appspot.com, *.*.appspot.com, appspot.com, not golang.org

Now, I want to trust this certificate myself (imagine a self-issued certificate where I can validate fingerprint etc.): how can I make a request and validate/trust the certificate?

I probably need to use openssl to download the certificate, load it into my file and fill tls.Config struct !?

Answer 1

Security note: Disabling security checks is dangerous and should be avoided

You can disable security checks globally for all requests of the default client:

package main

import (
    "fmt"
    "net/http"
    "crypto/tls"
)

func main() {
    http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
    _, err := http.Get("https://golang.org/")
    if err != nil {
        fmt.Println(err)
    }
}

You can disable security check for a client:

package main

import (
    "fmt"
    "net/http"
    "crypto/tls"
)

func main() {
    tr := &http.Transport{
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    }
    client := &http.Client{Transport: tr}
    _, err := client.Get("https://golang.org/")
    if err != nil {
        fmt.Println(err)
    }
}

Answer 2

Proper way (as of Go 1.13) (provided by answer below):

customTransport := http.DefaultTransport.(*http.Transport).Clone()
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: customTransport}

Original Answer:

Here's a way to do it without losing the default settings of the DefaultTransport, and without needing the fake request as per user comment.

defaultTransport := http.DefaultTransport.(*http.Transport)

// Create new Transport that ignores self-signed SSL
customTransport := &http.Transport{
  Proxy:                 defaultTransport.Proxy,
  DialContext:           defaultTransport.DialContext,
  MaxIdleConns:          defaultTransport.MaxIdleConns,
  IdleConnTimeout:       defaultTransport.IdleConnTimeout,
  ExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,
  TLSHandshakeTimeout:   defaultTransport.TLSHandshakeTimeout,
  TLSClientConfig:       &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: customTransport}

Shorter way:

customTransport := &(*http.DefaultTransport.(*http.Transport)) // make shallow copy
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: customTransport}

Warning: For testing/development purposes only. Anything else, proceed at your own risk!!!

Answer 3

All of these answers are wrong! Do not use InsecureSkipVerify to deal with a CN that doesn't match the hostname. The Go developers unwisely were adamant about not disabling hostname checks (which has legitimate uses - tunnels, nats, shared cluster certs, etc), while also having something that looks similar but actually completely ignores the certificate check. You need to know that the certificate is valid and signed by a cert that you trust. But in common scenarios, you know that the CN won't match the hostname you connected with. For those, set ServerName on tls.Config. If tls.Config.ServerName == remoteServerCN, then the certificate check will succeed. This is what you want. InsecureSkipVerify means that there is NO authentication; and it's ripe for a Man-In-The-Middle; defeating the purpose of using TLS.

There is one legitimate use for InsecureSkipVerify: use it to connect to a host and grab its certificate, then immediately disconnect. If you setup your code to use InsecureSkipVerify, it's generally because you didn't set ServerName properly (it will need to come from an env var or something - don't belly-ache about this requirement... do it correctly).

In particular, if you use client certs and rely on them for authentication, you basically have a fake login that doesn't actually login any more. Refuse code that does InsecureSkipVerify, or you will learn what is wrong with it the hard way!

Answer 4

The correct way to do this if you want to maintain the default transport settings is now (as of Go 1.13):

customTransport := http.DefaultTransport.(*http.Transport).Clone()
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client = &http.Client{Transport: customTransport}

Transport.Clone makes a deep copy of the transport. This way you don't have to worry about missing any new fields that get added to the Transport struct over time.

Answer 5

If you want to use the default settings from http package, so you don't need to create a new Transport and Client object, you can change to ignore the certificate verification like this:

tr := http.DefaultTransport.(*http.Transport)
tr.TLSClientConfig.InsecureSkipVerify = true

Answer 6

Generally, The DNS Domain of the URL MUST match the Certificate Subject of the certificate.

In former times this could be either by setting the domain as cn of the certificate or by having the domain set as a Subject Alternative Name.

Support for cn was deprecated for a long time (since 2000 in RFC 2818) and Chrome browser will not even look at the cn anymore so today you need to have the DNS Domain of the URL as a Subject Alternative Name.

RFC 6125 which forbids checking the cn if SAN for DNS Domain is present, but not if SAN for IP Address is present. RFC 6125 also repeats that cn is deprecated which was already said in RFC 2818. And the Certification Authority Browser Forum to be present which in combination with RFC 6125 essentially means that cn will never be checked for DNS Domain name.