107

Given a binary, compiled with Go using GOOS=linux and GOARCH=amd64, deployed to a docker container based on alpine:3.3, the binary will not run if the docker engine host is Ubuntu (15.10):

sh: /bin/artisan: not found

This same binary (compiled for the same OS and arch) will run just fine if the docker engine host is busybox (which is the base for alpine) deployed within a VirtualBox VM on Mac OS X.

This same binary will also run perfectly fine if the container is based on one of Ubuntu images.

Any idea what this binary is missing?

This is what I've done to reproduce (successful run in VirtualBox/busybox on OS X not shown):

Build (building explicitly with flags even though the arch matches):

➜  artisan git:(master) ✗ GOOS=linux GOARCH=amd64 go build

Check it can run on the host:

➜  artisan git:(master) ✗ ./artisan 
10:14:04.925 [ERROR] artisan: need a command, one of server, provision or build

Copy to docker dir, build, run:

➜  artisan git:(master) ✗ cp artisan docker/build/bin/        
➜  artisan git:(master) ✗ cd docker 
➜  docker git:(master) ✗ cat Dockerfile 
FROM docker:1.10
COPY build/ /
➜  docker git:(master) ✗ docker build -t artisan .
Sending build context to Docker daemon 10.15 MB
Step 1 : FROM docker:1.10
...
➜  docker git:(master) ✗ docker run -it artisan sh
/ # /bin/artisan 
sh: /bin/artisan: not found

Now changing the image base to phusion/baseimage:

➜  docker git:(master) ✗ cat Dockerfile 
#FROM docker:1.10
FROM phusion/baseimage
COPY build/ /
➜  docker git:(master) ✗ docker build -t artisan .
Sending build context to Docker daemon 10.15 MB
Step 1 : FROM phusion/baseimage
...
➜  docker git:(master) ✗ docker run -it artisan sh
# /bin/artisan
08:16:39.424 [ERROR] artisan: need a command, one of server, provision or build
Oleg Sklyar
  • 9,834
  • 6
  • 39
  • 62
  • 5
    Does adding CGO_ENABLED=0 help? – Martin Gallagher Mar 29 '16 at 08:42
  • Magic, it does. Could you please elaborate in an answer and I will accept. – Oleg Sklyar Mar 29 '16 at 08:44
  • Could you please try `go build -tags netgo -a -v std` with CGO_ENABLED=1? I think it could be issues with the net package, causing dynamic linking. – Martin Gallagher Mar 29 '16 at 08:57
  • As you suggested this helped `CGO_ENABLED=1 go build -tags netgo -a -v`. If I understand correctly in contrast to CGO_ENABLED=0 this will also preserve the TLS functionality in the net package, which otherwise would be lost, is that right? Can I see somehow what is linked into the binary statically and what is left for dynamic linking? – Oleg Sklyar Mar 29 '16 at 09:05
  • 2
    By default CGO can be used for the net package - using the above tag or CGO_ENABLED=0 forces the Go std implementation for lookups to be used - what you can do is do: `ldd output.bin` on each build variant to see if they're truly statically compiled or if there's any dynamic linking going on. – Martin Gallagher Mar 29 '16 at 09:09
  • Cool, thanks a lot for help and explanation! – Oleg Sklyar Mar 29 '16 at 09:17
  • When you use CGO, the binary is linked against glibc, but alpine uses musl libc. (also, the tls package doesn't use cgo, except to get root certs on darwin) – JimB Mar 29 '16 at 12:57
  • This seems similar to the issue that I had and solved here: http://stackoverflow.com/questions/34729748/installed-go-binary-not-found-in-path-on-alpine-linux-docker/35613430#35613430 – sheldonk Mar 29 '16 at 21:26
  • @sheldonk Thanks for the tip. I will give it a try even though linking everything statically would be my preference. – Oleg Sklyar Mar 29 '16 at 22:25
  • There are also some alpine glibc base images that you could try out that should solve your issue – sheldonk Mar 29 '16 at 23:07

7 Answers7

119

By default, if using the net package a build will likely produce a binary with some dynamic linking, e.g. to libc. You can inspect dynamically vs. statically link by viewing the result of ldd output.bin

There are two solutions I've come across:

  • Disable CGO, via CGO_ENABLED=0
  • Force the use of the Go implementation of net dependencies, netgo via go build -tags netgo -a -v, this is implemented for a certain platforms

From https://golang.org/doc/go1.2:

The net package requires cgo by default because the host operating system must in general mediate network call setup. On some systems, though, it is possible to use the network without cgo, and useful to do so, for instance to avoid dynamic linking. The new build tag netgo (off by default) allows the construction of a net package in pure Go on those systems where it is possible.

The above assumes that the only CGO dependency is the standard library's net package.

Martin Gallagher
  • 4,444
  • 2
  • 28
  • 26
  • 6
    `CGO_ENABLED=0` also fixed my problem: Trying to run alpine-built go program on non-alpine docker. The error in my case just said `docker: Error response from daemon: Container command not found or does not exist..` – Vlad A. Ionescu Apr 21 '16 at 14:03
  • For anyone using Bazel, the above can be accomplished with the `--features=static --features=pure` flags. – Ben Elgar Apr 16 '18 at 13:51
  • Thanks for this answer, took a lot of Googling to find. – Kyle Gibbons Nov 24 '18 at 23:35
  • I think another optional solution is copy source code into image and exec go build in image. – g10guang Apr 19 '19 at 03:20
  • CGO_ENABLED=0 should fix the problem. This condition is important too, GOARCH=amd64 – Naren Yellavula Dec 29 '19 at 18:54
  • 1
    It would be great if this answer also included what possible issues the 2 solutions could cause. Are either of them pretty much guaranteed to be safe? Or could they cause other runtime issues when the program tries to execute some of the related code? I'm building someone else's project as a component of my code, so I don't have a ton of insight into the related code that could rely on CGO. – Stephen Smith Sep 11 '20 at 18:19
105

I had the same issue with a go binary, and I got it to work after adding this to my docker file:

RUN apk add --no-cache libc6-compat
mtlynch
  • 3,403
  • 4
  • 31
  • 31
zakaria amine
  • 3,412
  • 2
  • 20
  • 35
  • 1
    help me to run CGO in my Alpine image – xsor Sep 26 '18 at 21:38
  • 2
    Saved a lot of time. +1 – Manwal May 31 '19 at 11:16
  • 3
    Unfortunately, this doesn't work for me, and gives `error relocating ...: fprintf chk: symbol not found` – TheDiveO Jun 01 '19 at 12:33
  • 1
    you need to build and run in the same environment. e.g if you build the binary in your local machine and run it inside docker, it may cause the issue – zakaria amine Jun 01 '19 at 17:58
  • 1
    I am running someone else's go binary in an Alpine docker container, and I don't want to recompile it myself. I'd rather use the publicly available binary. This solution fixes the issue and lets me run the go binary in an Alpine docker container – Brandon Jul 26 '19 at 06:19
  • Thanks a lot, does it matter where the above command is placed in the Docker-file. Say after which command? – TrevorDeTutor Aug 16 '22 at 08:35
  • glad it helps. as long as it's installed before running the binary, it should be ok – zakaria amine Aug 17 '22 at 09:24
  • 1
    After a struggle of 3 hours, this worked!!! Thanks – Ashwani Oct 12 '22 at 18:44
  • 1
    Just for the record - in case you are using multi-stage builds in your Dockerfile, this command is required in the stage where the binary is **run**, not built. That's where it will otherwise fail finding the required dynamic libraries. – Beolap Nov 28 '22 at 05:52
11

Go compiler from your build machine probably links your binary with libraries on different location than in Alpine. In my case it was compiled with dependencies under /lib64 but Alpine does not use that folder.

FROM alpine:edge AS build
RUN apk update
RUN apk upgrade
RUN apk add --update go=1.8.3-r0 gcc=6.3.0-r4 g++=6.3.0-r4
WORKDIR /app
ENV GOPATH /app
ADD src /app/src
RUN go get server # server is name of our application
RUN CGO_ENABLED=1 GOOS=linux go install -a server

FROM alpine:edge
WORKDIR /app
RUN cd /app
COPY --from=build /app/bin/server /app/bin/server
CMD ["bin/server"]

I'm working on article about this issue. You can find draft with this solution here https://kefblog.com/Post/2017-07-04_golang-in-docker-without-disabling-cgo-on-alpine .

Norbert Szydlik
  • 111
  • 1
  • 4
11

What did the trick for me was enabling static linking in the linker options:

$ go build -ldflags '-linkmode external -w -extldflags "-static"'

The -linkmode option tells Go to use the external linker, the -extldflags option sets options to pass to the linker and the -w flag disables DWARF debug info to improve binary size.

See go tool link and Statically compiled Go programs, always, even with cgo, using musl for more details

msiemens
  • 2,223
  • 1
  • 28
  • 38
  • 1
    That works however, its maybe worth pointing out that its also throwing a warning. => `warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking` I guess in case of alpine base image its ok though. – The Fool Jun 28 '21 at 12:02
3

While executing a go binary inside a debian docker container, faced this issue: /bin/bash: line 10: /my/go/binary: No such file or directory

The binary was built by using docker-in-docker (dind) from an alpine container using command: GOOS=linux GOARCH=amd64 go build

Fixed it by using following env while building the binary: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build

varunrocks6
  • 31
  • 1
3

I had an app that required CGO_ENABLED=1.

The fix for me to run the compiled go binary in a debian-slim container was to build the binary using RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -o goapp

And run the following commands in the debian slim

RUN apt-get update && apt-get install -y musl-dev
RUN ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1

Made me able to run the goapp afterwards

TIP: ldd goapp showed that libc.musl-x86_64 was missing in the container.

sirius
  • 200
  • 6
  • 14
0

The reason for this problem is that the alpine image uses musl libc, and the binary program compiled by cgo relies on gnu libc, so there will be compatibility problems. You can use the ldd command to view the dynamic libraries that a binary depends on.

root# ldd main 
 !10807         
  linux-vdso.so.1 (0x00007ffea7fab000)
  libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f7b1f818000)
  libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7b1f5f9000)
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7b1f208000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f7b1fa32000)

There are ways around this, go build sets CGO_ENABLED to 0.

env CGO_ENABLED=0 go build -o main main.go

In addition, if you insist on using cgo to compile the program, you can use frolvlad/alpine-glibc as the base image to solve this problem.

FROM frolvlad/alpine-glibc
WORKDIR /hello
COPY . .
CMD ["./main"]

hope it can help you.

SafeCar
  • 1
  • 3