Golang slice append vs assign performance

Question

To make slice append operation faster we need to allocate enough capacity. There's two ways to append slice, Here is the code:

func BenchmarkSliceAppend(b *testing.B) {
    a := make([]int, 0, b.N)
    for i := 0; i < b.N; i++ {
        a = append(a, i)
    }
}

func BenchmarkSliceSet(b *testing.B) {
    a := make([]int, b.N)
    for i := 0; i < b.N; i++ {
        a[i] = i
    }
}

And the result is:

BenchmarkSliceAppend-4 200000000 7.87 ns/op 8 B/op 0 allocs/op

BenchmarkSliceSet-4 300000000 5.76 ns/op 8 B/op

Why is a[i] = i faster than a = append(a, i)?

It's good to know that the classic assignment by index is faster. I think that the `append` way is strange and error-prone. ‍♂️ — Ferran Maylinch, Apr 08 '21 at 15:54
What if the size of the slice could be up to `<=N` - unknown until runtime - surely that has an impact. Therefore, the declaration of the slice differs. — Mark, Jan 10 '23 at 17:23

icza · Accepted Answer · 2016-07-29T15:12:28.350

30

a[i] = i simply assigns the value i to a[i]. This is not appending, it's just a simple assignment.

Now the append:

a = append(a, i)

In theory the following happens:

This calls the builtin append() function. For that, it first has to copy the a slice (slice header, backing array is not part of the header), and it has to create a temporary slice for the variadic parameter which will contain the value i.
Then it has to reslice a if it has enough capacity (it has in your case) like a = a[:len(a)+1] - which involves assigning the new slice to a inside the append().
_{(If a would not have big enough capacity to do the append "in-place", a new array would have to be allocated, content from slice copied, and then the assign / append be executed - but it is not the case here.)}
Then assigns i to a[len(a)-1].
Then returns the new slice from append(), and this new slice is assigned to the local variable a.

A lot of things happen here compared to a simple assignment. Even if many of these steps are optimized and / or inlined, as a minimum addition to assigning i to an element of the slice, the local variable a of slice type (which is a slice header) has to be updated in each cycle of the loop.

Recommended reading: The Go Blog: Arrays, slices (and strings): The mechanics of 'append'

edited Jul 29 '16 at 15:12

answered Jul 29 '16 at 09:08

icza

389,944
63
907
827

is there a way to append to a slice without reassigning it? – Dec 21 '18 at 04:08
given that append does a lot of things, is it worth to perform `copy(a,b)` followed by `if len(a) – Sep 16 '19 at 10:16
2

@mh-cbon It really depends on how critical the performance is. `append()` may be clearer and more readable, which is also important. If every nanosecond counts, then maybe. Should be measured, and properly documented if a less-readable version is chosen. – icza Sep 16 '19 at 10:35
Have I interpreted this correctly that go makes no use of [realloc](https://man7.org/linux/man-pages/man3/realloc.3p.html) or similar? It seems strange that a builtin append() function would not. – Philip Couling Apr 07 '22 at 12:13
1

@PhilipCouling If the current slice has enough capacity, it is just resliced and the new elements assigned to elements of the resliced slice. If the current slice does not have enough capacity, then a new backing array does get allocated (not optionally), and elements from the old slice('s backing array) are copied over. No reallocation happens, as this is a documented behavior, and could break existing, legal apps that depend on this behavior (that the new backing array does not share memory with the old one). – icza Apr 07 '22 at 12:38
@icza that's the statement I was looking for ;-). Thanks very much for your time. – Philip Couling Apr 07 '22 at 12:46

score 13 · Answer 2 · answered Sep 03 '18 at 10:58

13

It seems like some improvements of Go compiler or runtime have been introduced since this question has been posted, so now (Go 1.10.1) there is no significant difference between append and direct assignment by index.

Also, I had to change your benchmarks slightly because of OOM panics.

package main

import "testing"

var result []int

const size = 32

const iterations = 100 * 1000 * 1000

func doAssign() {
    data := make([]int, size)
    for i := 0; i < size; i++ {
        data[i] = i
    }
    result = data
}

func doAppend() {
    data := make([]int, 0, size)
    for i := 0; i < size; i++ {
        data = append(data, i)
    }
    result = data
}

func BenchmarkAssign(b *testing.B) {
    b.N = iterations
    for i := 0; i < b.N; i++ {
        doAssign()
    }
}

func BenchmarkAppend(b *testing.B) {
    b.N = iterations
    for i := 0; i < b.N; i++ {
        doAppend()
    }
}

Results:

➜  bench_slice_assign go test -bench=Bench .
goos: linux
goarch: amd64
BenchmarkAssign-4       100000000           80.9 ns/op
BenchmarkAppend-4       100000000           81.9 ns/op
PASS
ok      _/home/isaev/troubles/bench_slice_assign    16.288s

answered Sep 03 '18 at 10:58

Vitaly Isaev

5,392
6
45
64

3

This is great news, but keep in mind that if `size` was larger and an initial array capacity isn't specified, the story is vastly different. – Parth Mehrotra Nov 12 '18 at 05:47
@ParthMehrotra what different? Can you please tell your opinion? Thanks. – sgon00 Mar 02 '19 at 09:41
2

Resizing an array is a costly operation. If you don't know the initial size of the array, those reallocations will make the whole operation take much longer. – Parth Mehrotra Mar 04 '19 at 16:28
5

this benchmark is dubious as to what it does measure. I think it measures allocations cost, not the speed comparison of the two different patterns. – Oct 20 '19 at 16:58
2

Cannot reproduce your results on go1.16.4 with 100K `size`. – Mitar May 30 '21 at 03:59

Golang slice append vs assign performance

2 Answers2

Linked