GoLang - memory allocation - []byte vs string

Question

In the below code:

c := "fool"
d := []byte("fool")
fmt.Printf("c: %T, %d\n", c, unsafe.Sizeof(c)) // 16 bytes
fmt.Printf("d: %T, %d\n", d, unsafe.Sizeof(d)) // 24 bytes

To decide the datatype needed to receive JSON data from CloudFoundry, am testing above sample code to understand the memory allocation for []byte vs string type.

Expected size of string type variable c is 1 byte x 4 ascii encoded letter = 4 bytes, but the size shows 16 bytes.

For byte type variable d, GO embeds the string in the executable program as a string literal. It converts the string literal to a byte slice at runtime using the runtime.stringtoslicebyte function. Something like... []byte{102, 111, 111, 108}

Expected size of byte type variable d is again 1 byte x 4 ascii values = 4 bytes but the size of variable d shows 24 bytes as it's underlying array capacity.

Why the size of both variables is not 4 bytes?

icza · Accepted Answer · 2021-04-16T20:33:15.387

11

Both slices and strings in Go are struct-like headers:

reflect.SliceHeader:

type SliceHeader struct {
        Data uintptr
        Len  int
        Cap  int
}

reflect.StringHeader:

type StringHeader struct {
        Data uintptr
        Len  int
}

The sizes reported by unsafe.Sizeof() are the sizes of these headers, exluding the size of the pointed arrays:

Sizeof takes an expression x of any type and returns the size in bytes of a hypothetical variable v as if v was declared via var v = x. The size does not include any memory possibly referenced by x. For instance, if x is a slice, Sizeof returns the size of the slice descriptor, not the size of the memory referenced by the slice.

To get the actual ("recursive") size of some arbitrary value, use Go's builtin testing and benchmarking framework. For details, see How to get memory size of variable in Go?

For strings specifically, see String memory usage in Golang. The complete memory required by a string value can be computed like this:

var str string = "some string"

stringSize := len(str) + int(unsafe.Sizeof(str))

edited Apr 16 '21 at 20:33

answered Apr 29 '19 at 13:56

icza

389,944
63
907
827

So, can I consider this answer as invalid? https://stackoverflow.com/a/26975872/3317808 – overexchange Apr 29 '19 at 14:05
3

@overexchange Again, as stressed in my answer, the value returned by `unsafe.Sizeof()` does not include any memory possibly referenced by the value. This means the memory needed to hold the actual bytes of a slice or string is not covered by it. – icza Apr 29 '19 at 14:07
I understand... my question is.. Is it not important to comment this [answer](https://stackoverflow.com/a/26975872/3317808) as invalid? – overexchange Apr 29 '19 at 14:24
2

@overexchange It depends on how you define the "size". If you need the referenced / pointed size too, then it's not valid. – icza Apr 29 '19 at 14:28
That answer says: "It returns the size in bytes, occupied by the value you pass into it." – overexchange Apr 29 '19 at 14:29
2

@overexchange Yes, because a slice value or a string value is just the header. This is the memory required to hold a slice or string header. But if you want to know how much memory it needs to allocate / create them, that requires to include the size of the array these headers point to. – icza Apr 29 '19 at 14:31
This throws an error in the go playground. https://play.golang.org/p/ugZoV5PeE9H – sprut Apr 16 '21 at 17:47
@sprut Thanks, fixed. – icza Apr 16 '21 at 20:33

GoLang - memory allocation - []byte vs string

1 Answers1

Linked