Does Go automatically close resources if not explicitly closed?

Question

The following Open followed by a deferred Close is idiomatic in Go:

func example() {
    f, err := os.Open(path)
    if err != nil {
        return
    }
    defer f.Close()
}

What happens if I don't have defer f.Close()?

When I call this function and f goes out of scope, does it automatically close the file or do I have a zombie file handle?

If it closes automatically, when exactly does it do this?

Answer 1

It is true Files are closed when garbage collected, but... as mentioned in "Mystery of finalizers in Go" from Alexander Morozov -- LK4D4math:

In Go we have both GC and pro-users :)
So, in my opinion explicit call of Close is always better than magic finalizer.

Alexander adds:

The problem with finalizers is that you have no control over them, and, more than that, you’re not expecting them.

Look at this code:

func getFd(path string) (int, error) {
    f, err := os.Open(path)
    if err != nil {
        return -1, err
    }
    return f.Fd(), nil
}

It’s pretty common operation to get file descriptor from path when you’re writing some stuff for linux.
But that code is unreliable, because when you’re return from getFd(), f loses its last reference and so your file is doomed to be closed sooner or later (when next GC cycle will come).

Here, the problem is not that file will be closed, but that it is not documented and not expected at all.

---

There was a proposal to extend the finalizer and detecting leaks (like file descriptor leaks)

But... Russ Cox quashed that down convincingly:

Anyone interested in this topic should read Hans Boehm's paper "Destructors, Finalizers, and Synchronization".
It greatly influenced our decision to limit the scope of finalizers in Go as much as possible.
They are a necessary evil for allowing reclamation of non-(heap memory) resources at the same time as associated heap memory, but they are inherently much more limited in their capabilities than most people initially believe.

We will not be expanding the scope of finalizers, either in implementation or in the standard library, or in the x repos, nor will we encourage others to expand that scope.

If you want to track manually-managed objects, you'd be far better off using runtime/pprof.NewProfile.

For example, inside Google's source tree we have a Google-wide "file" abstraction, and the Go wrapper package for this declares a global:

var profiles = pprof.NewProfile("file")

The end of the function that creates a new File says:

profiles.Add(f, 2)
return f

and then f.Close does

profiles.Remove(f)

Then we can get a profile of all in-use files, "leaked" or otherwise, from /debug/pprof/file or from pprof.Lookup("file").WriteTo(w, 0).
And that profile includes stack traces.

Answer 2

Go applications are expected to explicitly release resources. There is no language feature that will implicitly release a resource when a variable goes out of scope. The language does provide the defer feature to make it easier to write the explicit code.

As bserdar and VonC noted, the garbage collector has a hook for releasing external resources (see runtime.SetFinalizer). This hook is used by the os.File type. Applications should not rely on this hook because it's not specified when objects are collected, if at all.

Answer 3

Files will be closed automatically when os.File is garbage collected. This appears to be done by SetFinalizer calls so the file will be closed eventually, not immediately after it becomes unreachable.

Answer 4

In your simple example, the defer statement is not required. The file is closed at the time the function scope is exited.

If, however, you return the variable f or save it in a global, its reference counter is incremented and thus it doesn't get deleted.

Here is an example to prove that the finalizer gets called immediately (it's garbage collected immediately because the reference counter reaches zero) in the reverse order they were initialized:

package main

import (
        "fmt"
        "runtime"
        "time"
)

type Foo struct {
        name    string
        num     int
}

func  finalizer(f *Foo) {
        fmt.Println("a finalizer has run for ", f.name, f.num)
}

var counter int
func MakeFoo(name string) (a_foo *Foo) {
        a_foo = &Foo{name, counter}
        counter++
        runtime.SetFinalizer(a_foo, finalizer)
        return
}

func Bar() {
        f1 := MakeFoo("one")
        f2 := MakeFoo("two")

        fmt.Println("f1 is: ", f1.name)
        fmt.Println("f2 is: ", f2.name)
}

func main() {
        for i := 0; i < 3; i++ {
                Bar()
                time.Sleep(time.Second)
                runtime.GC()
        }
        fmt.Println("done.")
}

The fact is that you have no real control over whether the finalizer gets called. But if you make sure to not copy the local variable anywhere, then it happens immediately.

Actually, an interesting fact is that if you use defer, then you create a reference to the object (you need to have a reference to be able to call the Close() at the time you quit your function).

This is a very important point since, if you have a loop, you'd actually make it much worse:

    ...
    for i:=0; i<10; i++ {
        f := os.Open(...)
        defer f.Close()
        ...
    } // without the defer, the file is closed here, as expected

} // <- defer is called here, so you will open 10 files
  // and keep all 10 open until this line

So you have to be pretty careful about the defer. There are situations where it doesn't work.