6

What happens if I return a slice of an array that is a local variable of a function or method? Does Go copy the array data into a slice create with make()? Will the capacity match the slice size or the array size?

func foo() []uint64 {
    var tmp [100]uint64
    end := 0
    ...
    for ... {
        ...
        tmp[end] = uint64(...)
        end++
        ...
    }
    ... 
    return tmp[:end]
} 
icza
  • 389,944
  • 63
  • 907
  • 827
chmike
  • 20,922
  • 21
  • 83
  • 106

3 Answers3

7

This is detailed in Spec: Slice expressions.

The array will not be copied, but instead the result of the slice expression will be a slice that refers to the array. In Go it is perfectly safe to return local variables or their addresses from functions or methods, the Go compiler performs an escape analysis to determine if a value may escape the function, and if so (or rather if it can't prove that a value may not escape), it allocates it on the heap so it will be available after the function returns.

The slice expression: tmp[:end] means tmp[0:end] (because a missing low index defaults to zero). Since you didn't specify the capacity, it will default to len(tmp) - 0 which is len(tmp) which is 100.

You can also control the result slice's capacity by using a full slice expression, which has the form:

a[low : high : max]

Which sets the resulting slice's capacity to max - low.

More examples to clarify the resulting slice's length and capacity:

var a [100]int

s := a[:]
fmt.Println(len(s), cap(s)) // 100 100
s = a[:50]
fmt.Println(len(s), cap(s)) // 50 100
s = a[10:50]
fmt.Println(len(s), cap(s)) // 40 90
s = a[10:]
fmt.Println(len(s), cap(s)) // 90 90

s = a[0:50:70]
fmt.Println(len(s), cap(s)) // 50 70
s = a[10:50:70]
fmt.Println(len(s), cap(s)) // 40 60
s = a[:50:70]
fmt.Println(len(s), cap(s)) // 50 70

Try it on the Go Playground.

Avoiding heap allocation

If you want to allocate it on the stack, you can't return any value that points to it (or parts of it). If it would be allocated on the stack, there would be no guarantee that after returning it remains available.

A possible solution to this would be to pass a pointer to an array as an argument to the function (and you may return a slice designating the useful part that the function filled), e.g.:

func foo(tmp *[100]uint64) []uint64 {
    // ...
    return tmp[:end]
}

If the caller function creates the array (on the stack), this will not cause a "reallocation" or "moving" to the heap:

func main() {
    var tmp [100]uint64
    foo(&tmp)
}

Running go run -gcflags '-m -l' play.go, the result is:

./play.go:8: leaking param: tmp to result ~r1 level=0
./play.go:5: main &tmp does not escape

The variable tmp is not moved to heap.

Note that [100]uint64 is considered a small array to be allocated on the stack. For details see What is considered "small" object in Go regarding stack allocation?

Community
  • 1
  • 1
icza
  • 389,944
  • 63
  • 907
  • 827
  • That is a nicely detailed answer. What confused me is my assumption that the array would be allocated on the stack and released on return like in C and C++. If it's allocated on the heap I gain nothing by using this code pattern. Is there another solution than using a recursive function to use the stack and to allocate once and only once the exactly required size ? – chmike Mar 01 '17 at 11:40
  • @chmike See edited answer. You may choose to pass a pointer to an array as an argument. – icza Mar 01 '17 at 11:48
  • Thanks for the suggestion. But it doesn't fit my goal. The function foo is generating some data to be returned in a slice which I know will never exceed 100 for instance. The problem is that I don't know in advance the amount of data produced and in general it will be much less than 100. Around 10 or so. I assumed the array was allocated on the stack. This assumption is wrong. Small arrays are. Not such big arrays. – chmike Mar 01 '17 at 12:07
  • @chmike You may return a slice of the array you passed that designates the _useful part (that you filled), that's not a problem. Edited to make it clear. Also this is considered a small array and it will be allocated on stack. For details see [What is considered “small” object in Go regarding stack allocation?](http://stackoverflow.com/questions/42243197/what-is-considered-small-object-in-go-regarding-stack-allocation/42244087#42244087) – icza Mar 01 '17 at 12:10
  • I tested in the playground as you suggested me and the array was apparently not allocated on the stack. It was with the array `[11]byte` which I used in another place of my code. I suspect that go has a strategy to keep stacks small because of goroutines. This would make sense. – chmike Mar 01 '17 at 12:15
  • @chmike If you read the linked answer, it shows the limit is around 64 KB after which variables are allocated on heap (even without escaping). `[100]uint64` is nowehere near that limit. If you return a slice pointing to it, that counts as "escaping". – icza Mar 01 '17 at 12:17
  • Check the code I wrote. Even an 11 uint64 array is apparently allocated on the stack. Replace uint64 with byte and you will see it's allocated on the stack (apparently). – chmike Mar 01 '17 at 12:21
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/136957/discussion-between-icza-and-chmike). – icza Mar 01 '17 at 12:30
  • @icza what does "leaking param: tmp to result ~r1 level=0" actually mean? – human Jan 07 '20 at 05:56
  • 1
    @bigdatamann That's the analysis result of the `foo()` function: it says that the analyzer detected that the (input) parameter `tmp` is referenced by the return value, so this should be taken into consideration when analysing calls of `foo()`. – icza Jan 07 '20 at 07:12
2

The data is not copied. The array will be used as the underlying array of the slice.

It looks like you're worrying about the life time of the array, but the compiler and garbage collector will figure that out for you. It's as safe as returning pointers to "local variables".

Art
  • 19,807
  • 1
  • 34
  • 60
  • I assumed the array was allocated on the stack. This is the case only for small arrays. The big array above is allocate on the heap, so indeed, a slice would carry it along and much memory space would be wasted unless I copy the relevant data in a slice allocated my self. But that function would allocate two objects, which is would like to avoid. So using that array is a bad choice in the first place. – chmike Mar 01 '17 at 12:12
2

Nothing wrong happens.

Go does not make a copy but compiler performs escape analysis and allocates the variable that will be visible outside function on heap.

The capacity will be the capacity of underlying array.

Grzegorz Żur
  • 47,257
  • 14
  • 109
  • 105