Go libstd.so with errors on Alpine

Question

I'm trying to build a Go executable with shared libraries. In Ubuntu, which uses GNU libc, it works. But when I try to use the same procedure on Alpine (Docker image golang:alpine, or 1.14.1-alpine3.11), which uses MUSL libc, the generated libstd.so is broken. Afterwards, if I try to compile the executable, the compilation fails.

This is the procedure:

$ docker run -it golang:alpine sh
/go # apk add --update alpine-sdk
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/38) Installing fakeroot (1.24-r0)
(2/38) Installing sudo (1.8.31-r0)
(3/38) Installing libcap (2.27-r0)
(4/38) Installing pax-utils (1.2.4-r0)
(5/38) Installing openssl (1.1.1d-r3)
(6/38) Installing libattr (2.4.48-r0)
(7/38) Installing attr (2.4.48-r0)
(8/38) Installing libacl (2.2.53-r0)
(9/38) Installing tar (1.32-r1)
(10/38) Installing pkgconf (1.6.3-r0)
(11/38) Installing patch (2.7.6-r6)
(12/38) Installing libgcc (9.2.0-r4)
(13/38) Installing libstdc++ (9.2.0-r4)
(14/38) Installing lzip (1.21-r0)
(15/38) Installing nghttp2-libs (1.40.0-r0)
(16/38) Installing libcurl (7.67.0-r0)
(17/38) Installing curl (7.67.0-r0)
(18/38) Installing abuild (3.5.0-r0)
Executing abuild-3.5.0-r0.pre-install
(19/38) Installing binutils (2.33.1-r0)
(20/38) Installing libmagic (5.37-r1)
(21/38) Installing file (5.37-r1)
(22/38) Installing gmp (6.1.2-r1)
(23/38) Installing isl (0.18-r0)
(24/38) Installing libgomp (9.2.0-r4)
(25/38) Installing libatomic (9.2.0-r4)
(26/38) Installing mpfr4 (4.0.2-r1)
(27/38) Installing mpc1 (1.1.0-r1)
(28/38) Installing gcc (9.2.0-r4)
(29/38) Installing musl-dev (1.1.24-r2)
(30/38) Installing libc-dev (0.7.2-r0)
(31/38) Installing g++ (9.2.0-r4)
(32/38) Installing make (4.2.1-r2)
(33/38) Installing fortify-headers (1.1-r0)
(34/38) Installing build-base (0.5-r1)
(35/38) Installing expat (2.2.9-r1)
(36/38) Installing pcre2 (10.34-r1)
(37/38) Installing git (2.24.1-r0)
(38/38) Installing alpine-sdk (1.0-r0)
Executing busybox-1.31.1-r9.trigger
OK: 196 MiB in 53 packages
/go # go install -a -buildmode=shared -linkshared std
/go # ldd /usr/local/go/pkg/linux_amd64_dynlink/libstd.so 
    /lib/ld-musl-x86_64.so.1 (0x7f689af41000)
    libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f689af41000)
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_malloc: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_realloc: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_free: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: main.main: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_stack_end: symbol not found
/go # go version
go version go1.14.1 linux/amd64
/go # go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build202809585=/tmp/go-build -gno-record-gcc-switches"
/go #

It works as expected when I do the same procedure with the same Golang version, but other distribution that has GNU libc:

$ docker run -it golang:buster sh
# go install -a -buildmode=shared -linkshared std
# ldd /usr/local/go/pkg/linux_amd64_dynlink/libstd.so
    linux-vdso.so.1 (0x00007ffe54bea000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f12f79a3000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f12f7982000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f12f77c1000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f12fa0d2000)
# go version
go version go1.14.1 linux/amd64
# go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build609530787=/tmp/go-build -gno-record-gcc-switches"
#

Am I missing some detail? Does Golang or Alpine have a bug?

I searched "[go] musl libc" here on SO and found this, perhaps it'll help: https://stackoverflow.com/questions/54524785/how-to-build-a-go-executable-that-doesnt-link-to-musl-libc — Adrian, Mar 26 '20 at 14:55
Try this [solution](https://stackoverflow.com/a/50861580/12354911) — Eldar, Mar 26 '20 at 15:16
@Adrian, it's not exactly the same scenario, but I'll try to do a "cross-compilation" on Ubuntu with MUSL to see what happens. Thanks. — Alexei Aguiar, Mar 26 '20 at 15:37

valiano · Answer 1 · 2020-05-12T18:11:15.727

It seems the Go runtime has some explicit GNU libc dependencies, as demonstrated by the link errors:

Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_malloc: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_realloc: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_free: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: main.main: symbol not found
Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: __libc_stack_end: symbol not found

Lightweight glibc compatibility (installing the libc6-compat package) probably won't do in this case, since it mostly add stubs, and some glibc functionalities will still be missing.

Trying to find the offending Go module, picking in the Go build cache following the go install -a -buildmode=shared -linkshared std (object files under.cache/go-build, looking for glibc dependencies with grep -R libc), one large object file stood out. Listing objects symbols with nm revealed it was the one consuming the required symbols:

        U __libc_free
        U __libc_malloc
        U __libc_realloc
        U __libc_stack_end

That object turned out to be race.go.

Google search revealed that Alpine libc compatibility is indeed a known issue for race, documented here:
https://github.com/golang/go/issues/14481

Luckily, the ticket mentions several proposed workarounds, such as building compiler-rt from source.

Thank you very much. I wouldn't find this conclusion by myself. I'll take a look at the proposed workarounds and try them. — Alexei Aguiar, Apr 02 '20 at 23:58
It almost worked by using the proposed Dockerfile in the issue thread. There is just one missing symbol. But the thread is going with recent news and I'll follow up the events. It seems that they're close to have the fix. ```ldd /usr/local/go/pkg/linux_amd64_dynlink/libstd.so /lib/ld-musl-x86_64.so.1 (0x7f03e5e0b000) libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f03e5e0b000) Error relocating /usr/local/go/pkg/linux_amd64_dynlink/libstd.so: main.main: symbol not found``` — Alexei Aguiar, Apr 03 '20 at 02:58

Go libstd.so with errors on Alpine

1 Answers1