13

I just started learning go, while going through slice tricks, couple of points are very confusing. can any one help me to clarify.

To cut elements in slice its given

Approach 1:

a = append(a[:i], a[j:]...)

but there is a note given that it may cause to memory leaks if pointers are used and recommended way is

Approach 2:

copy(a[i:], a[j:])
for k, n := len(a)-j+i, len(a); k < n; k++ {
    a[k] = nil // or the zero value of T
}
a = a[:len(a)-j+i]

Can any one help me understand how memory leaks happen. I understood sub slice will be backed by the main array. My thought is irrespective of pointer or not we have to follow approach 2 always.

update after @icza and @Volker answer..

Lets say you have a struct 

type Books struct {
    title   string
    author  string
}

var Book1 Books
var Book2 Books 

    /* book 1 specification */
    Book1.title = "Go Programming"
    Book1.author = "Mahesh Kumar"

    Book2.title = "Go Programming"
    Book2.author = "Mahesh Kumar"

    var bkSlice = []Books{Book1, Book2}
    var bkprtSlice = []*Books{&Book1, &Book2}

now doing 

bkSlice = bkSlice[:1]

bkSlice still holds the Book2 in backing array which is still in memory and is not required to be. so do we need to do

bkSlice[1] = Books{}

so that it will be GCed. I understood pointers have to be nil-ed as the slice will hold unnecessary references to the objects outside backing array.

Samarendra
  • 760
  • 10
  • 23
  • 1
    This is not a memory leak in the traditional sense: Once the backing array gets garbage collected because no slice reference it any longer all memory is claimed and no leak happens. It is just that the memory is claimed only after all slices are gone. With the recommended code the memory pointed to can be claimed _earlier_. – Volker Mar 07 '19 at 14:17
  • See: [Does go garbage collect parts of slices?](https://stackoverflow.com/questions/28432658/does-go-garbage-collect-parts-of-slices/28432812#28432812) – icza Mar 07 '19 at 14:28
  • @icza i saw this answer, even in that is says if we use pointers it may leak memory..my understanding is even if we use direct object this will cause memory issues. why it causes specifically for pointers. – Samarendra Mar 07 '19 at 14:36
  • @Volker I agree with you, but why it is specifically mentioned in case of pointer. it should be the case weather its pointers or not? – Samarendra Mar 07 '19 at 14:37
  • Again: There is never any memory leak. It is about allowing to claim memory earlier. This early claim can be done is the slice contains pointers (or slices or maps or channels) which can be nil-ed in the backing array before slicing. If you have a slice of ints the ints cannot be GCed as they are part of the backing array. – Volker Mar 07 '19 at 15:01

1 Answers1

18

Simplest can be demonstrated by a simple slice expression.

Let's start with a slice of *int pointers:

s := []*int{new(int), new(int)}

This slice has a backing array with a length of 2, and it contains 2 non-nil pointers, pointing to allocated integers (outside of the backing array).

Now if we reslice this slice:

s = s[:1]

Length will become 1. The backing array (holding 2 pointers) is not touched, it sill holds 2 valid pointers. Even though we don't use the 2nd pointer now, since it is in memory (it is the backing array), the pointed object (which is a memory space for storing an int value) cannot be freed by the garbage collector.

The same thing happens if you "cut" multiple elements from the middle. If the original slice (and its backing array) was filled with non-nil pointers, and if you don't zero them (with nil), they will be kept in memory.

Why isn't this an issue with non-pointers?

Actually, this is an issue with all pointer and "header" types (like slices and strings), not just pointers.

If you would have a slice of type []int instead of []*int, then slicing it will just "hide" elements that are of int type which must stay in memory as part of the backing array regardless of if there's a slice that contains it or not. The elements are not references to objects stored outside of the array, while pointers refer to objects being outside of the array.

If the slice contains pointers and you nil them before the slicing operation, if there are no other references to the pointed objects (if the array was the only one holding the pointers), they can be freed, they will not be kept due to still having a slice (and thus the backing array).

Update:

When you have a slice of structs:

var bkSlice = []Books{Book1, Book2}

If you slice it like:

bkSlice = bkSlice[:1]

Book2 will become unreachabe via bkSlice, but still will be in memory (as part of the backing array).

You can't nil it because nil is not a valid value for structs. You can however assign its zero value to it like this:

bkSlice[1] = Book{}
bkSlice = bkSlice[:1]

Note that a Books struct value will still be in memory, being the second element of the backing array, but that struct will be a zero value, and thus will not hold string references, thus the original book author and title strings can be garbage collected (if no one else references them; more precisely the byte slice referred from the string header).

The general rule is "recursive": You only need to zero elements that refer to memory located outside of the backing array. So if you have a slice of structs that only have e.g. int fields, you do not need to zero it, in fact it's just unnecessary extra work. If the struct has fields that are pointers, or slices, or e.g. other struct type that have pointers or slices etc., then you should zero it in order to remove the reference to the memory outside of the backing array.

icza
  • 389,944
  • 63
  • 907
  • 827