0

I want to discard all bytes over a given limit during reading of HTTP Response (skip too large response) using LimitedReader.
Is this the correct way to do that?

func getRequestData(req *http.Request, client *http.Client,
    responseSizeLimit int64) ([]byte, error) {
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }
    defer func() {
        io.Copy(ioutil.Discard, resp.Body) // response body must be read to the end and closed
        resp.Body.Close()
    }()

    buffer := bufio.NewReader(resp.Body)
    reader := io.LimitReader(buffer, responseSizeLimit)

    return ioutil.ReadAll(reader)
}

And what if I want to return io.Reader instead of []byte, is it possible for http.Response.Body and limitedReader ?

Vladimir Bershov
  • 2,701
  • 2
  • 21
  • 51
  • Yes, this works, have you encountered a problem? Also, yes you can replace the `resp.Body` with any `io.ReadCloser` you need. – JimB Jun 07 '22 at 18:30
  • @JimB How can I combine the necessary `defer resp.Body.Close()` and `return reader`? – Vladimir Bershov Jun 07 '22 at 18:37
  • 2
    If you return an io.ReadCloser it is generally understood that the caller is supposed to call Close when done reading. You can state that explicitly in the docs for the function if you wish, but since this is an internal (i.e. unexported) function anyway, and you therefore necessarily know all call sites, there's little point to that. – Peter Jun 07 '22 at 18:52
  • @EmilePels seems I cannot return some kind of limited reader from the func because then will be not possible to read bytes remaining over the limit from the Body before Close – Vladimir Bershov Jun 07 '22 at 19:08
  • 1
    You can create your own `io.ReadCloser` which drains the connection on `Close()`. However, if you are trying to skip responses that are too large, then you normally want to drop the connection if there's data left, since you don't know how much there could be, so it's usually better to just stop reading and call `Close()`. – JimB Jun 07 '22 at 19:16

0 Answers0