When to use embed?

Question

Starting Go 1.16, we have the embed directive. It helps us embed an extra file (say, a .txt file) into the executable without having to supply that file additionally. (Reference, here).

I don't quite follow what construes an 'extra file'. Are all files not ending in .go construed as extra files? Any exceptions?

I want to ship a binary protobuf definition file as a part of the main binary so that my code can read it. Would this file be an extra file? Or would it be a part of the main binary itself?

Use embed when you want to access a file in the package source tree at runtime. If you want to read the protobuf def file at runtime, then embed the def file. — Charlie Tumahai, Jun 17 '21 at 03:28
@CeriseLimón, sorry did not get you. My question is, in case of a binary protobuf def file (that is being read), would it automatically be packaged into the main binary, or would I have to use `embed` to explicitly pack it? — Someone, Jun 17 '21 at 03:32
`embed` allows you to include non-compiled files in the binary. For instance you can embed the text .proto file. — Burak Serdar, Jun 17 '21 at 03:34

score 4 · Accepted Answer · 2021-06-17T04:11:50.553

It is often the case that a running Go program does not have access to the file system directories from which the the program was compiled.

The embed feature slurps up files in the package source code tree at build time and makes those files available at runtime.

Packages should embed files from the package source code tree that the package needs to access at runtime.

Here's an example: A package foo has has the source file foo.go. The package needs to access the file abc.txt in the same directory as foo.go.

If the current working directory is the source code code directory, then the package code can access the file using os.Open or anything that calls os.Open:

  f, err := os.Open("abc.txt")
  if err != nil {
      // handle error
  }
  defer f.Close()
  // do something with f

This code can fail. The current working directory at runtime is not necessarily the same as the package source code directory. It usually is not the same directory. The package source directory may not even be available to the running application.

In this case, the package should embed the file to ensure that the package can access the contents of the file:

//go:embed abc.txt
var abc []byte

I am afraid this does not answer my question. Suppose I have a file `main.go` (in package `main`) that refers another file `foo.go` (in package `foo`). These two packages (`main` and `foo`) would both be present in the compiled binary. However, suppose `foo.go` reads from `abc.txt` - would this file `abc.txt` be present in the binary (so that `foo.go` can read it at runtime)? Or would I have to use `embed` to add it to the binary? — Someone, Jun 17 '21 at 03:41

score 3 · Answer 2 · answered Jun 17 '21 at 04:06

Embed allows you to treat non-Go files as a true part of the source tree, rather than as a run-time dependency. You can see embedded files as an extension of your source code, and you can treat them similarly (mainly that the contents of the file are assured by your usual source control measures).

With normal files, even if they are in theory a static element of the application, they still generally require error checking and validation every time the program is run, or else inexplicable errors may ensue. With an embedded file, you don't have to worry about this once all tests are passed and you deploy the application.

The main issues of an embedded file are that you can't change the contents without rebuilding the program, and the (potentially large) file size gets added to the binary. If that doesn't pose a problem, go ahead.

Any time you can reasonably embed a file instead of loading it at run time, it is probably desirable to do so as it makes program deployment simpler.

score 0 · Answer 3 · answered Jun 17 '21 at 11:23

To add to the other answers, one shortcoming of the embed directive is that you cannot use it to embed OS-level resources, such as Windows application manifests or an icon for display next to the binary in Windows Explorer; at least not in a way that the OS will pick up on it.

On Windows, you would use something like goversioninfo to embed those.

When to use embed?

3 Answers3

Linked