How to remove unused code at compile time?

Question

We've built a Go package that is used by many of us.

It's imported using the standard import ("package-name") method.

At compile time though, all of our utilities, including the very small ones, end up as very large binaries.

We've extracted all the strings in the utilities and discovered that the entire package is being compiled into each and every utility. Including functions that are not being used by those utilities.

EDIT 1:

Thank you to the people who are responding to this question.

Here's what we are seeing:

main.go

package main

import "play/subplay"

func main() {
    subplay.A()
}

play/subplay.go

package subplay

func A() {
    fmt.Printf("this is function A()")
}

func B() {
    fmt.Printf("secret string")
}

Function B() is never called. Yet, after building the binary, we find the string "secret string" into main.exe.

How can we remove unused code from Go programs at compile time?

What version of Go are you using? Because the `"secret string"` does **not** appear in the executable for me if `subplay.B()` is not called (Go 1.8, targeting both linux and windows)! — icza, Mar 16 '17 at 09:59

score 22 · Accepted Answer · edited May 23 '17 at 12:17

The compiler already does this. All code ends up in package files (.a), but in the executable binary the Go tool does not include everything from imported packages, only what is needed (or more precisely: it excludes things that it can prove unreachable).

See possible duplicate Splitting client/server code.

One thing to note here: if an imported package (from which you only want to include things you refer to) imports other packages, this has to be done recursively of course.

For example if you import this package:

package subplay

func A() {}

And call nothing from it:

package main

import _ "play/subplay"

func main() {
}

The result binary (Go 1.8, linux, amd64) will be 960,134 bytes (roughly 1 MB).

If you change subplay to import net/http:

package subplay

import _ "net/http"

func A() {}

But still don't call anything from net/http, the result will be: 5,370,935 bytes (roughly 5 MB)! (Note that net/http also imports 39 other packages!)

Now if you go ahead and start using things from net/http:

package subplay

import "net/http"

func A() {
    http.ListenAndServe("", nil)
}

But in the main package you still don't call subplay.A(), the size of the executable binary does not change: remains 5,370,935 bytes!

And when you change the main package to call subplay.A():

package main

import "play/subplay"

func main() {
    subplay.A()
}

The result binary grows: 5,877,919 bytes!

If you change http.ListenAndServe() to http.ListenAndServeTLS():

func A() {
    http.ListenAndServeTLS("", "", "", nil)
}

Result binary is: 6,041,535 bytes.

As you can see, what gets compiled into the executable binary does depend on what you call / use from the imported packages.

Also don't forget that Go is a statically linked language, the result executable binary has to include everything it needs. See related question: Reason for huge size of compiled executable of Go

Hi Icza, thank you for responding. We're seeing different results on our end. See an updated message above with the details. Thank you in advance for looking into this! — ABC, Mar 16 '17 at 09:49
It's worth noting that importing a package, whether or not you use it, causes package variables to be initialized and the `init()` function to run. These can reference other parts of the package, which can in turn reference others... So, just because you don't reference anything in the package, does not mean that importing it leaves the whole package unused, or that the compiler will be able to elide it. — Adrian, May 31 '17 at 22:03
@Adrian Yes, and this is exactly what my second example ought to prove: nothing from `subplay` is used, yet the executable binary is increased by 4 MB because it references `net/http`. — icza, May 31 '17 at 22:46

Zoyd · Answer 2 · 2017-03-16T10:52:43.537

-1

You can turn your package into a plugin, if you don’t mind the restrictions (plugins need Go 1.8 and currently only work on Linux).

[edit] Since you are using Windows, this is not an option for you.

edited Mar 16 '17 at 10:52

answered Mar 16 '17 at 06:59

Zoyd

3,449
1
18
27

And every plugin itself is statically linked and thus would be even bigger. – Sarath Sadasivan Pillai Mar 16 '17 at 07:13
Extracting the package into a plugin means that the package binary code will only be in one .so file, not in the utility executables. Thus, that code is removed from the utilities, as requested. Of course the plugin itself is statically linked, but the utilities load it dynamically, they don’t import it. – Zoyd Mar 16 '17 at 08:04
1

While plugins may help in some cases to reduce the size of executable binary+plugin files, they may also increase it! For example if the plugin imports `fmt` and so does your main app, the `fmt` package will be included both in the executable binary and in the plugin file! Not to mention it's a lot less convenient to load and use a plugin than to import and use a package. Also not to mention performance... – icza Mar 16 '17 at 08:11
They nevertheless achieve the goal of not including unused (or used, for that matter) code from the package in each and every command that uses it. – Zoyd Mar 16 '17 at 08:22

How to remove unused code at compile time?

2 Answers2

Linked

Related