From io.Reader to string in Go

Question

I have an io.ReadCloser object (from an http.Response object).

What's the most efficient way to convert the entire stream to a string object?

Answer 1

EDIT:

Since 1.10, strings.Builder exists. Example:

buf := new(strings.Builder)
n, err := io.Copy(buf, r)
// check errors
fmt.Println(buf.String())

---

OUTDATED INFORMATION BELOW

The short answer is that it it will not be efficient because converting to a string requires doing a complete copy of the byte array. Here is the proper (non-efficient) way to do what you want:

buf := new(bytes.Buffer)
buf.ReadFrom(yourReader)
s := buf.String() // Does a complete copy of the bytes in the buffer.

This copy is done as a protection mechanism. Strings are immutable. If you could convert a []byte to a string, you could change the contents of the string. However, go allows you to disable the type safety mechanisms using the unsafe package. Use the unsafe package at your own risk. Hopefully the name alone is a good enough warning. Here is how I would do it using unsafe:

buf := new(bytes.Buffer)
buf.ReadFrom(yourReader)
b := buf.Bytes()
s := *(*string)(unsafe.Pointer(&b))

There we go, you have now efficiently converted your byte array to a string. Really, all this does is trick the type system into calling it a string. There are a couple caveats to this method:

1. There are no guarantees this will work in all go compilers. While this works with the plan-9 gc compiler, it relies on "implementation details" not mentioned in the official spec. You can not even guarantee that this will work on all architectures or not be changed in gc. In other words, this is a bad idea.
2. That string is mutable! If you make any calls on that buffer it will change the string. Be very careful.

My advice is to stick to the official method. Doing a copy is not that expensive and it is not worth the evils of unsafe. If the string is too large to do a copy, you should not be making it into a string.

Answer 2

Answers so far haven't addressed the "entire stream" part of the question. I think the good way to do this is ioutil.ReadAll. With your io.ReaderCloser named rc, I would write,

Go >= v1.16

if b, err := io.ReadAll(rc); err == nil {
    return string(b)
} ...

Go <= v1.15

if b, err := ioutil.ReadAll(rc); err == nil {
    return string(b)
} ...

Answer 3

data, _ := ioutil.ReadAll(response.Body)
fmt.Println(string(data))

Answer 4

func copyToString(r io.Reader) (res string, err error) {
    var sb strings.Builder
    if _, err = io.Copy(&sb, r); err == nil {
        res = sb.String()
    }
    return
}

Answer 5

The most efficient way would be to always use []byte instead of string.

In case you need to print data received from the io.ReadCloser, the fmt package can handle []byte, but it isn't efficient because the fmt implementation will internally convert []byte to string. In order to avoid this conversion, you can implement the fmt.Formatter interface for a type like type ByteSlice []byte.

Answer 6

var b bytes.Buffer
b.ReadFrom(r)

// b.String()

Answer 7

I like the bytes.Buffer struct. I see it has ReadFrom and String methods. I've used it with a []byte but not an io.Reader.