14

I was optimizing a code using a map[string]string where the value of the map was only either "A" or "B". So I thought Obviously a map[string]bool was way better as the map hold around 50 millions elements.

var a = "a"
var a2 = "Why This ultra long string take the same amount of space in memory as 'a'"
var b = true
var c map[string]string
var d map[string]bool

c["t"] = "A"
d["t"] = true

fmt.Printf("a: %T, %d\n", a, unsafe.Sizeof(a))
fmt.Printf("a2: %T, %d\n", a2, unsafe.Sizeof(a2))
fmt.Printf("b: %T, %d\n", b, unsafe.Sizeof(b))
fmt.Printf("c: %T, %d\n", c, unsafe.Sizeof(c))
fmt.Printf("d: %T, %d\n", d, unsafe.Sizeof(d))
fmt.Printf("c: %T, %d\n", c, unsafe.Sizeof(c["t"]))
fmt.Printf("d: %T, %d\n", d, unsafe.Sizeof(d["t"]))

And the result was:

a: string, 8
a2: string, 8
b: bool, 1
c: map[string]string, 4
d: map[string]bool, 4
c2: map[string]string, 8
d2: map[string]bool, 1

While testing I found something weird, why a2 with a really long string use 8 bytes, same as a which has only one letter?

Jason Aller
  • 3,541
  • 28
  • 38
  • 38
Dylan
  • 197
  • 1
  • 2
  • 10

1 Answers1

23

unsafe.Sizeof() does not recursively go into data structures, it just reports the "shallow" size of the value passed. Quoting from its doc:

The size does not include any memory possibly referenced by x. For instance, if x is a slice, Sizeof returns the size of the slice descriptor, not the size of the memory referenced by the slice.

Maps in Go are implemented as pointers, so unsafe.Sizeof(somemap) will report the size of that pointer.

Strings in Go are just headers containing a pointer and a length. See reflect.StringHeader:

type StringHeader struct {
        Data uintptr
        Len  int
}

So unsafe.Sizeof(somestring) will report the size of the above struct, which is independent of the length of the string value (which is the value of the Len field).

To get the actual memory requirement of a map ("deeply"), see How much memory do golang maps reserve? and also How to get memory size of variable in Go?

Go stores the UTF-8 encoded byte sequences of string values in memory. The builtin function len() reports the byte-length of a string, so basically the memory required to store a string value in memory is:

var str string = "some string"

stringSize := len(str) + int(unsafe.Sizeof(str))

Also don't forget that a string value may be constructed by slicing another, bigger string, and thus even if the original string is no longer referenced (and thus no longer needed), the bigger backing array will still be required to be kept in memory for the smaller string slice.

For example:

s := "some loooooooong string"
s2 := s[:2]

Here, even though memory requirement for s2 would be len(s2) + unsafe.Sizeof(str) = 2 + unsafe.Sizeof(str), still, the whole backing array of s will be retained.

icza
  • 389,944
  • 63
  • 907
  • 827
  • Thank you for your reply! So basicaly following the StringHeader struct, the size of a string in memory is equal to sizeof(uintptr) * Len ? – Dylan Oct 17 '18 at 10:05
  • 1
    @Dylan No, `StringHeader.Data` is a pointer where the bytes (characters) of the string are stored in memory. The size of a `string` value is basically its length (it's a byte-length), plus the size of the header. See edited answer. – icza Oct 17 '18 at 10:07
  • I get an error when trying to run the example https://play.golang.org/p/ugZoV5PeE9H – sprut Apr 16 '21 at 17:48
  • @sprut Thanks for the note, I fixed it (it just needs an `int` conversion). – icza Apr 16 '21 at 20:32
  • 1
    @sprut here is your example corrected https://play.golang.org/p/PyX257Efll1 – Guilherme Oliveira Jun 21 '21 at 11:22