Slice within a loop seems to retain the previous/last reference (depending on the length of slice)

Question

This seems very strange, With in a loop there is a local variable slice with new value assigned for each loop and I'm appending that slice to a global sliceWrappers. After the loop completes, all values inside the global slice contains only reference to the last value set on that local slice variable.

Code:

package main

import (
    "fmt"
    "strconv"
)

func main() {
    var sliceWrappers [][]string
    initialSlice := append([]string{}, "hi")
    initialSlice = append(initialSlice, "there")
    
    // Remove this line and it works fine
    initialSlice = append(initialSlice, "I'm")
    
    for i := 0; i < 2; i++ {
        slice := append(initialSlice, strconv.Itoa(i))
        fmt.Printf("Slice Value : %+v, Initial Value : %+v\n", slice, initialSlice)
        sliceWrappers = append(sliceWrappers, slice)
    }

    for _, sliceWrapper := range sliceWrappers {
        fmt.Printf("%+v\n", sliceWrapper)
    }
}

Actual Output:

Slice Value : [hi there I'm 0], Initial Value : [hi there I'm]
Slice Value : [hi there I'm 1], Initial Value : [hi there I'm]
[hi there I'm 1]
[hi there I'm 1]

Expected Output:

Slice Value : [hi there I'm 0], Initial Value : [hi there I'm]
Slice Value : [hi there I'm 1], Initial Value : [hi there I'm]
[hi there I'm 0]  <------ This is not happening
[hi there I'm 1]

If I Remove initialSlice = append(initialSlice, "I'm") line, then it works perfectly.

Slice Value : [hi there 0], Initial Value : [hi there]
Slice Value : [hi there 1], Initial Value : [hi there]
[hi there 0]  <------ Works Perfectly
[hi there 1]

I Believe this has something to do with the append

The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements.

If the above condition was responsible for it, then shouldn't value of initialSlice which was printed inside the loop also be same as slice?

Playground - https://play.golang.org/p/b3SDGoA2Lzv

PS : I unfortunately wrote test cases for my code with just 3 levels nesting and it passed fine. I now have to take care of copy for slices inside loops.

Answer 1

Slices are based around a pointer to an array, a length, and a capacity. You put slice in sliceWrappers on the first iteration (so it contains a pointer to an array). On the second iteration, the append(initialSlice, strconv.Itoa(i)) call changes the values in this same array, because the memory location hasn't changed. This array is pointed to by both slice in the first and second iteration, so both slices that end up in sliceWrappers point to the same data.

You can avoid this by copying the data to a new slice before adding it to sliceWrappers:

    for i := 0; i < 2; i++ {
        slice := append(initialSlice, strconv.Itoa(i))
        fmt.Printf("Slice Value : %+v, Initial Value : %+v\n", slice, initialSlice)
        copiedSlice := make([]string, len(slice))
        copy(copiedSlice, slice)
        sliceWrappers = append(sliceWrappers, copiedSlice)
    }

Which gives the expected output:

Slice Value : [hi there I'm 0], Initial Value : [hi there I'm]
Slice Value : [hi there I'm 1], Initial Value : [hi there I'm]
[hi there I'm 0]
[hi there I'm 1]

As for removing the line initialSlice = append(initialSlice, "I'm"): When you append to a slice, it will check whether it can fit the new length inside the capacity. If not, it will allocate a new array (and thereby new memory location). The shorter slice containing "hi there" is at its capacity and appending to it will allocate a new array and create a slice with a larger capacity.

• If you have the line initialSlice = append(initialSlice, "I'm") in your program, the new array will be allocated before the loop. The append(...)s inside the loop won't cause a new allocation.
• If you don't have the line in your program, the append(...)s inside the loop will cause a new allocation, so each ends up with a different memory location, which is why they don't overwrite each other.

My source is Go Slices: usage and internals https://blog.golang.org/slices-intro#TOC_4.

Answer 2

// Remove this line and it works fine
//initialSlice = append(initialSlice, "I'm")
fmt.Printf("Slice Value : %p - %d\n", initialSlice, cap(initialSlice))
...
for i := 0; i < 2; i++ {
    slice := append(initialSlice, strconv.Itoa(i))
    fmt.Printf("Slice Value : %p - %p\n", slice, initialSlice)
    ...
}

print the address and capacity of initialSlice and slice as above. when uncomment the append I'm line. It output as below:

Slice Value : 0xc00009c000 - 2
Slice Value : 0xc0000a6000 - 0xc00009c000
Slice Value : 0xc00009e040 - 0xc00009c000

and if comment the line, output below:

Slice Value : 0xc00009e000 - 4
Slice Value : 0xc00009e000 - 0xc00009e000
Slice Value : 0xc00009e000 - 0xc00009e000

And then why it output as you expect when comment the line?
because in this scene, the capacity of initialSlice is 2. A new underlying array will be allocated when append new elements for it dont have enough space to achieve append action.
And when you uncomment the line, the capacity of initialSlice is 4, it will modify the array in place.

reference: append doc

The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated. Append returns the updated slice. It is therefore necessary to store the result of append, often in the variable holding the slice itself: slice = append(slice, elem1, elem2) slice = append(slice, anotherSlice...) As a special case, it is legal to append a string to a byte slice, like this: slice = append([]byte("hello "), "world"...)