Specify timeout when tracing HTTP request in Go

Question

I know the usual method of specifying a timeout with HTTP requests by doing:

httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

However, I can't seem to figure out how to do the same when tracing HTTP requests. Here is the piece of code I am working with:

func timeGet(url string) (httpTimingBreakDown, error) {
    req, _ := http.NewRequest("GET", url, nil)

    var start, connect, dns, tlsHandshake time.Time
    var timingData httpTimingBreakDown
    timingData.url = url

    trace := &httptrace.ClientTrace{
        TLSHandshakeStart:    func() { tlsHandshake = time.Now() },
        TLSHandshakeDone:     func(cs tls.ConnectionState, err error) { timingData.tls = time.Since(tlsHandshake) },
    }

    req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
    start = time.Now()

    http.DefaultTransport.(*http.Transport).ResponseHeaderTimeout = time.Second * 10 // hacky way, worked earlier but don't work anymore

    if _, err := http.DefaultTransport.RoundTrip(req); err != nil {
        fmt.Println(err)
        return timingData, err
    }

    timingData.total = time.Since(start)

    return timingData, nil
}

I am firing this function inside a goroutine. My sample data set is 100 urls. All goroutines fire, but eventually the program ends in 30+ secs as if the timeout is 30secs.

Earlier I made the same to work by using the hacky way of changing the default inside of it to 10 secs and anything that took too long, timed out and the program ended at 10.xxx secs but now its taking 30.xx secs.

What would be a proper way of specifying a timeout in this scenario?

score 11 · Accepted Answer · edited Jun 20 '20 at 09:12

11

I know the usual method of specifying a timeout with HTTP requests by doing:
httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

Actually, the preferred method is to use a context.Context on the request. The method you've used is just a short-cut suitable for simple use cases.

req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
    return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()
req = req.WithContext(ctx)

And this method should work nicely for your situation as well.

edited Jun 20 '20 at 09:12

Community

1
1

answered Feb 21 '18 at 08:27

Jonathan Hall

75,165
16
143
189

Thank you! Any recommendations on keeping up with the latest recommended methods with newer Go versions of doing something? Often I land up on google results which are doing some hackery to accomplish something which is not needed anymore but I just don't know that. Makes sense? – Ashfame Feb 21 '18 at 08:30
@Ashfame: Read the release notes for every new Go release. – Jonathan Hall Feb 21 '18 at 08:33
@FlimzyThat's doable when I have gained some experience with the language but difficult when just starting out with a language. – Ashfame Feb 21 '18 at 08:42
@Ashfame: The release notes are pretty short for every version. It shouldn't be very intimidating at all. Skip over the parts that don't interest you. – Jonathan Hall Feb 21 '18 at 08:45
You can start [here](https://golang.org/doc/go1.10) with the Go 1.10 release notes which was released just recently. – Jonathan Hall Feb 21 '18 at 08:47
Yep checked that out. I guess it will start making sense sooner or later. But to start with, I don't even understand what I am looking to extract out of those release notes. Like, I am not up to speed up on things as I am just beginning, so what has changed is less significant for me vs a guide dictating how to do things properly. – Ashfame Feb 21 '18 at 08:50
@Ashfame: Well, you asked how to keep up to date with recommendations with newer versions of Go. There's really only one answer :) – Jonathan Hall Feb 21 '18 at 08:57
True, I guess I recognize this problem now that I have experience and starting with a new language. How would you answer this: What's a Comprehensive / Definitive guide to becoming a badass go developer? And you can only answer in definitive manner :P – Ashfame Feb 21 '18 at 09:00

Specify timeout when tracing HTTP request in Go

1 Answers1

Linked