How to read a binary file in Go

Question

I'm completely new to Go and I'm trying to read a binary file, either byte by byte or several bytes at a time. The documentation doesn't help much and I cannot find any tutorial or simple example (by the way, how could Google give their language such an un-googlable name?). Basically, how can I open a file, then read some bytes into a buffer? Any suggestion?

Answer 1

For manipulating files, the os package is your friend:

f, err := os.Open("myfile")
if err != nil {
   panic(err)
}
defer f.Close()

For more control over how the file is open, see os.OpenFile() instead (doc).

For reading files, there are many ways. The os.File type returned by os.Open (the f in the above example) implements the io.Reader interface (it has a Read() method with the right signature), it can be used directly to read some data in a buffer (a []byte) or it can also be wrapped in a buffered reader (type bufio.Reader).

Specifically for binary data, the encoding/binary package can be useful, to read a sequence of bytes into some typed structure of data. You can see an example in the Go doc here. The binary.Read() function can be used with the file read using the os.Open() function, since as I mentioned, it is a io.Reader.

And there's also the simple to use io/ioutil package, that allows you to read the whole file at once in a byte slice (ioutil.ReadFile(), which takes a file name, so you don't even have to open/close the file yourself), or ioutil.ReadAll() which takes a io.Reader and returns a slice of bytes containing the whole file. Here's the doc on ioutil.

Finally, as others mentioned, you can google about the Go language using "golang" and you should find all you need. The golang-nuts mailing list is also a great place to look for answers (make sure to search first before posting, a lot of stuff has already been answered). To look for third-party packages, check the godoc.org website.

HTH

Answer 2

This is what I use to read an entire binary file into memory

func RetrieveROM(filename string) ([]byte, error) {
    file, err := os.Open(filename)

    if err != nil {
        return nil, err
    }
    defer file.Close()

    stats, statsErr := file.Stat()
    if statsErr != nil {
        return nil, statsErr
    }

    var size int64 = stats.Size()
    bytes := make([]byte, size)

    bufr := bufio.NewReader(file)
    _,err = bufr.Read(bytes)

    return bytes, err
}

Answer 3

For example, to count the number of zero bytes in a file:

package main

import (
    "fmt"
    "io"
    "os"
)

func main() {
    f, err := os.Open("filename")
    if err != nil {
        fmt.Println(err)
        return
    }
    defer f.Close()
    data := make([]byte, 4096)
    zeroes := 0
    for {
        data = data[:cap(data)]
        n, err := f.Read(data)
        if err != nil {
            if err == io.EOF {
                break
            }
            fmt.Println(err)
            return
        }
        data = data[:n]
        for _, b := range data {
            if b == 0 {
                zeroes++
            }
        }
    }
    fmt.Println("zeroes:", zeroes)
}

Answer 4

You can't whimsically cast primitive types to (char*) like in C, so for any sort of (de)serializing of binary data use the encoding/binary package. http://golang.org/pkg/encoding/binary . I can't improve on the examples there.

Answer 5

Here is an example using Read method:

package main

import (
   "io"
   "os"
)

func main() {
   f, e := os.Open("a.go")
   if e != nil {
      panic(e)
   }
   defer f.Close()
   for {
      b := make([]byte, 10)
      _, e = f.Read(b)
      if e == io.EOF {
         break
      } else if e != nil {
         panic(e)
      }
      // do something here
   }
}

https://golang.org/pkg/os#File.Read