Build Docker Image From Go Code

Question

I'm trying to build a Docker image using the Docker API and Docker Go libraries (https://github.com/docker/engine-api/). Code example:

package main
import (
    "fmt"
    "github.com/docker/engine-api/client"
    "github.com/docker/engine-api/types"
    "golang.org/x/net/context"
)
func main() {
    defaultHeaders := map[string]string{"User-Agent": "engine-api-cli-1.0"}
    cli, err := client.NewClient("unix:///var/run/docker.sock", "v1.22", nil, defaultHeaders)
    if err != nil {
        panic(err)
    }
    fmt.Print(cli.ClientVersion())
    opt := types.ImageBuildOptions{
        CPUSetCPUs:   "2",
        CPUSetMems:   "12",
        CPUShares:    20,
        CPUQuota:     10,
        CPUPeriod:    30,
        Memory:       256,
        MemorySwap:   512,
        ShmSize:      10,
        CgroupParent: "cgroup_parent",
        Dockerfile:   "dockerSrc/docker-debug-container/Dockerfile",
    }
    _, err = cli.ImageBuild(context.Background(), nil, opt)
    if err == nil || err.Error() != "Error response from daemon: Server error" {
        fmt.Printf("expected a Server Error, got %v", err)
    }
}

The error is always same:

Error response from daemon: Cannot locate specified Dockerfile: dockerSrc/docker-debug-container/Dockerfile

or

Error response from daemon: Cannot locate specified Dockerfile: Dockerfile

Things I've checked:

The folder exists in build path
I tried both relative and absolute path
There are no softlinks in the path
I tried the same folder for binary and Dockerfile
docker build <path> works
and bunch of other stuff

My other option was to use RemoteContext which looks like it works, but only for fully self contained dockerfiles, and not the ones with "local file presence".

Update: Tried passing tar as buffer, but got the same result with the following:

  dockerBuildContext, err := os.Open("<path to>/docker-debug-    container/docker-debug-container.tar")
  defer dockerBuildContext.Close()

    opt := types.ImageBuildOptions{
        Context:      dockerBuildContext,
        CPUSetCPUs:   "2",
        CPUSetMems:   "12",
        CPUShares:    20,
        CPUQuota:     10,
        CPUPeriod:    30,
        Memory:       256,
        MemorySwap:   512,
        ShmSize:      10,
        CgroupParent: "cgroup_parent",
        //  Dockerfile:   "Dockerfile",
    }

    _, err = cli.ImageBuild(context.Background(), nil, opt)

score 14 · Answer 1 · answered Oct 02 '17 at 01:50

The following works for me;

package main

import (
    "archive/tar"
    "bytes"
    "context"
    "io"
    "io/ioutil"
    "log"
    "os"

    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
)

func main() {
    ctx := context.Background()
    cli, err := client.NewEnvClient()
    if err != nil {
        log.Fatal(err, " :unable to init client")
    }

    buf := new(bytes.Buffer)
    tw := tar.NewWriter(buf)
    defer tw.Close()

    dockerFile := "myDockerfile"
    dockerFileReader, err := os.Open("/path/to/dockerfile")
    if err != nil {
        log.Fatal(err, " :unable to open Dockerfile")
    }
    readDockerFile, err := ioutil.ReadAll(dockerFileReader)
    if err != nil {
        log.Fatal(err, " :unable to read dockerfile")
    }

    tarHeader := &tar.Header{
        Name: dockerFile,
        Size: int64(len(readDockerFile)),
    }
    err = tw.WriteHeader(tarHeader)
    if err != nil {
        log.Fatal(err, " :unable to write tar header")
    }
    _, err = tw.Write(readDockerFile)
    if err != nil {
        log.Fatal(err, " :unable to write tar body")
    }
    dockerFileTarReader := bytes.NewReader(buf.Bytes())

    imageBuildResponse, err := cli.ImageBuild(
        ctx,
        dockerFileTarReader,
        types.ImageBuildOptions{
            Context:    dockerFileTarReader,
            Dockerfile: dockerFile,
            Remove:     true})
    if err != nil {
        log.Fatal(err, " :unable to build docker image")
    }
    defer imageBuildResponse.Body.Close()
    _, err = io.Copy(os.Stdout, imageBuildResponse.Body)
    if err != nil {
        log.Fatal(err, " :unable to read image build response")
    }
}

loved that you used the native "archive/tar" library to generate the tar, this code is maintainable even more years from now on. — Daniel Andrei Mincă, Feb 07 '21 at 19:04

score 11 · Answer 2 · answered Aug 30 '16 at 13:36

@Mangirdas: staring at a screen long enough DOES help - at least in my case. I have been stuck with the same issue for some time now. You were right to use the tar file (your second example). If you look at the API doc here https://docs.docker.com/engine/reference/api/docker_remote_api_v1.24/#/build-image-from-a-dockerfile you can see that it expects a tar. What really helped me was looking at other implementations of the client, perl and ruby in my case. Both create a tar on the fly when being asked to build an image from a directory. Anyway, you only need to put your dockerBuildContext somewhere else (see the cli.ImageBuild())

dockerBuildContext, err := os.Open("/Path/to/your/docker/tarfile.tar")
defer dockerBuildContext.Close()

buildOptions := types.ImageBuildOptions{
    Dockerfile:   "Dockerfile", // optional, is the default
}

buildResponse, err := cli.ImageBuild(context.Background(), dockerBuildContext, buildOptions)
if err != nil {
    log.Fatal(err)
}
defer buildResponse.Body.Close()

I am not there with naming the images properly yet, but at least I can create them... Hope this helps. Cheers

I ended doing this :/ So looks like kinda way to do this now. — Mangirdas, Sep 15 '16 at 20:30
You do not have to call close on the dockerBuildContext; `defer dockerBuildContext.Close()` This is because, cli.ImageBuild calls go's net/http client; https://github.com/moby/moby/blob/master/client/image_build.go#L34 And the go client always closes the request body by default; https://github.com/golang/go/blob/c8aec4095e089ff6ac50d18e97c3f46561f14f48/src/net/http/client.go#L472 — Komu, Oct 02 '17 at 01:16

score 9 · Answer 3 · answered Oct 02 '18 at 23:43

9

The Docker package has a function for creating a TAR from a file path. It's whats used by the CLI. It's not in the client package so it need to be installed separately:

import (
    "github.com/mitchellh/go-homedir"
    "github.com/docker/docker/pkg/archive"
)

func GetContext(filePath string) io.Reader {
    // Use homedir.Expand to resolve paths like '~/repos/myrepo'
    filePath, _ := homedir.Expand(filePath)
    ctx, _ := archive.TarWithOptions(filePath, &archive.TarOptions{})
    return ctx
}

cli.ImageBuild(context.Background(), GetContext("~/repos/myrepo"), types.ImageBuildOptions{...})

answered Oct 02 '18 at 23:43

kylieCatt

10,672
5
43
51

1

This is a great answer because it's important that your application respects the '.dockerignore' file. – Kevin Minehart Apr 28 '22 at 17:13
1

You can get the list of dockerignore patterns from https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerignore/dockerignore.go and provide that in the TarOptions. – Kevin Minehart Apr 28 '22 at 17:22

score 5 · Answer 4 · answered Mar 06 '17 at 23:34

I agree with Marcus Havranek's answer, that method has worked for me. Just want to add how to add a name to an image, since that seemed like an open question:

buildOptions := types.ImageBuildOptions{
    Tags:   []string{"imagename"},
}

Hope this helps!

score 5 · Answer 5 · answered Nov 07 '19 at 06:13

Combining a few of the answers, and adding how to correctly parse the returned JSON using DisplayJSONMessagesToStream.

package main

import (
    "os"
    "log"

    "github.com/docker/docker/api/types"
    "github.com/docker/docker/pkg/archive"
    "github.com/docker/docker/pkg/jsonmessage"
    "github.com/docker/docker/pkg/term"
    "golang.org/x/net/context"
)

// Build a dockerfile if it exists
func Build(dockerFilePath, buildContextPath string, tags []string) {

    ctx := context.Background()
    cli := getCLI()

    buildOpts := types.ImageBuildOptions{
        Dockerfile: dockerFilePath,
        Tags:       tags,
    }

    buildCtx, _ := archive.TarWithOptions(buildContextPath, &archive.TarOptions{})

    resp, err := cli.ImageBuild(ctx, buildCtx, buildOpts)
    if err != nil {
        log.Fatalf("build error - %s", err)
    }
    defer resp.Body.Close()

    termFd, isTerm := term.GetFdInfo(os.Stderr)
    jsonmessage.DisplayJSONMessagesStream(resp.Body, os.Stderr, termFd, isTerm, nil)
}

I've left our a few convenience functions like getCLI but I'm sure you have your own equivalents.

Thanks for this. I'm wondering why you chose to write to Stderr though? Why not Stdout? — Steve, Aug 22 '20 at 02:25

score 3 · Answer 6 · answered Jul 05 '18 at 02:42

I encounter same problem. Finally find out the tar file should be docker build context even with Dockerfile.

Here is my code,

package main

import (
    "log"
    "os"

    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
    "golang.org/x/net/context"
)

func main() {
    dockerBuildContext, err := os.Open("/Users/elsvent/workspace/Go/src/test/test.tar")
    defer dockerBuildContext.Close()

    buildOptions := types.ImageBuildOptions{
        SuppressOutput: true,
        PullParent:     true,
        Tags:           []string{"xxx"},
        Dockerfile:     "test/Dockerfile",
    }
    defaultHeaders := map[string]string{"Content-Type": "application/tar"}
    cli, _ := client.NewClient("unix:///var/run/docker.sock", "v1.24", nil, defaultHeaders)
    buildResp, err := cli.ImageBuild(context.Background(), dockerBuildContext, buildOptions)
    if err != nil {
    log.Fatal(err)
    }
    defer buildResp.Body.Close()
}

score 0 · Answer 7 · answered Mar 27 '21 at 01:46

0

opt := types.ImageBuildOptions{
    Dockerfile: "Dockerfile",
}

filePath, _ = homedir.Expand(".")
buildCtx, _ := archive.TarWithOptions(filePath, &archive.TarOptions{})

x, err := cli.ImageBuild(context.Background(), buildCtx, opt)
io.Copy(os.Stdout, x.Body)

answered Mar 27 '21 at 01:46

user15491526

1

1

go get github.com/mitchellh/go-homedir – user15491526 Mar 27 '21 at 01:50

Build Docker Image From Go Code

7 Answers7

Linked