Is it necessary to read and close the response body to reuse the TCP connection on retry?

Question

I am implementing a retry using http.RoundTripper in Go. Here is an implementation example.

type retryableRoundTripper struct {
    tr            http.RoundTripper
    maxRetryCount int
}

func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
    for count := 0; count < t.maxRetryCount; count++ {
        log.Printf("retryableRoundTripper retry: %d\n", count+1)
        resp, err = t.tr.RoundTrip(req)

        if err != nil || resp.StatusCode != http.StatusTooManyRequests {
            return resp, err
        }
    }

    return resp, err
}

func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
    for count := 0; count < t.maxRetryCount; count++ {
        log.Printf("retryableRoundTripper retry: %d\n", count+1)
        resp, err = t.tr.RoundTrip(req)

        if err != nil || resp.StatusCode != http.StatusTooManyRequests {
            return resp, err
        }
    }

    // add
    io.Copy(ioutil.Discard, resp.Body)
    resp.Body.Close()

    return resp, err
}

As a side note, I've written a test and have confirmed that retries work as expected. (In Go Playground, it times out, but it works locally.)

https://play.golang.org/p/08YWV0kjaKr

https://stackoverflow.com/questions/17948827/reusing-http-connections-in-golang — whitespace, Aug 06 '20 at 05:01

JimB · Accepted Answer · 2020-08-06T14:01:46.593

Of course you need to read the connection to ensure that it can be reused, and Closing the connection is required as documented.

As stated in the docs:

The client must close the response body when finished with it

and

The default HTTP client's Transport may not reuse HTTP/1.x "keep-alive" TCP connections if the Body is not read to completion and closed.

If the server wants to send more data than fits in the initial read buffers, it is going to be blocked sending the response. This means that if the transport were to attempt to send a new request over that connection the server may not be able to handle it because it never completed the first request. This will usually result in a client error of connection reset by peer and a server error of write: broken pipe.

If you want to make an attempt to reuse the connection, but limit the amount read, use an io.LimitedReader and/or check the ContentLength value. This way you can discard the connection when it's faster to handle the errors and bring up a new connection than to read an unbounded amount of data. See Limiting amount of data read in the response to a HTTP GET request.

Thank you! I understand. The official documentation quotes were also very helpful. https://golang.org/src/net/http/response.go#L35 — Tsuji Daishiro, Aug 07 '20 at 00:25

Is it necessary to read and close the response body to reuse the TCP connection on retry?

1 Answers1