Trying to better understand the behavior of slices

Question

When I read over slices in Go they seemed reasonable enough. I understand that the slice structure has a capacity based on the underlying array and a length of the currently contained elements, also that the slice references the array underneath.

However, when I was playing around with Go's "A Tour of Go", I couldn't understand why the following would decrease the underlying arrays capacity.

package main

import "fmt"

func main() {
    s := []int{2, 3, 5, 7, 11, 13}
    printSlice(s)

    s = s[1:5]
    printSlice(s)

    s = s[:0]
    printSlice(s)

    s = s[0:5]
    printSlice(s)
}

func printSlice(s []int) {
    fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}

Result:

len=6 cap=6 [2 3 5 7 11 13]
len=4 cap=5 [3 5 7 11]
len=0 cap=5 []
len=5 cap=5 [3 5 7 11 13]

Why did the capacity change between the first and second call of printSlice(s)?

Answer 1

The underlying array has a length of six initially, which is also the capacity of the slice.

When you reslice with s = s[1:5] you are effectively "ignoring" the first element of the underlying array, so you are left with five elements, and that's the new capacity of the slice.

Originally:

| <------- array -------> |              
| 2 | 3 | 5 | 7 | 11 | 13 |
| <------- slice -------> |              

After reslicing:

| <------- array -------> |          
| 2 | 3 | 5 | 7 | 11 | 13 |
    | <----- slice -----> |          

The array underlying a slice may extend past the end of the slice. The capacity is a measure of that extent: it is the sum of the length of the slice and the length of the array beyond the slice.

https://golang.org/ref/spec#Slice_types

Note that it says beyond the slice (in this case the length of the array beyond the slice is zero). Any array elements that may exist before the beginning of the slice do not count towards the slice's capacity. In fact, these elements are completely unreachable, unless another slice exists that references this region of the array.

To keep the capacity at six, Go would have to create a new array of size six and copy the last five elements of the original array to the beginning of the new array, but reslicing never changes the underlying array.

Answer 2

As Peter said, reslicing—whether using the old s[low:high] syntax, or the new s[low:high:max] syntax—never changes the underlying array itself. (The Go specification calls the new syntax with the max expression added a full slice expression.)

The main thing to watch out for with slices also helps you to think about using slices. Remember that with any slice, there are really two parts:

• there is a slice header, which keeps a pointer to the array and provides the length and capacity; and
• there is some underlying array, somewhere.

You can provide the underlying array yourself:

var space [10]int

someSlice := space[:]

or you can use make (or a function that itself calls make) to have the runtime allocate the array somehow.¹ Meanwhile, the compiler will allocate the slice header for you.

Calling any function, passing the slice as an argument, will pass it a copy of the slice header. The underlying array is still wherever it is, but the function you call gets a copy of the slice header. Note that in your own functions, you can inspect this slice header, directly or indirectly; see icza's answer to How to inspect slice header?.

Functions that you call might decide on their own: Gosh, this slice header talks about an array that's too small for the things I would like to put into the array. I'll allocate a new array somewhere, copy all the old array's values via the old slice header, and use the new slice header with the new array to do my work. The built in append does exactly this.

A function that does do this sort of thing tends to do it not always, but only sometimes: sometimes, the slice header talks about an array that's already big enough, so it can just put more stuff in the array.

A function that does this will, in general, return for you a new slice header. You should grab this new value and use it. The new slice header will have a new pointer, length, and capacity. The new pointer may be the same as the old pointer! If so, watch out for this or any other function—perhaps an outer or inner recursive call—that tries to use either the old or the new slice header to modify the underlying array, because that other function might not expect that modification.

If append (or whatever function you called) decided that, gee, the old underlying array was too small, and therefore allocated a new one and copied data to it, whoever still has the old slice header is "safe" from any meddling anyone does to the underlying array using the new slice header. So the code might sometimes work, and then sometimes fail, depending on whether append (or whatever) decided to re-use the existing array, or make a new one.

That's what you need to watch out for, and that's why you need to pay attention to who has which slice headers and what underlying arrays those headers might be using.

---

¹There's a certain degree of "magic" behind this make allocation, which in the end, is just some very careful use of the unsafe package so that the compiler and runtime can collude to produce useful results. You are allowed to peek at the internals as the Go runtime source code is available to all, but the Go authors reserve the right to change the internals in the future, if the update makes Go code faster or better or whatever.