Build docker images in parallel using docker Go client

Question

I'm using Docker's Go client to build my projects. This post highlights how to do that with the Go client. I'm calling ImageBuild on three of my Dockerfiles (1.Dockerfile, 2.Dockerfile, and 3.Dockerfile) as a test. Here is my code:

func GetContext(filePath string) io.Reader {
    // Use homedir.Expand to resolve paths like '~/repos/myrepo'
    filePath, _ = homedir.Expand(filePath)
    ctx, err := archive.TarWithOptions(filePath, &archive.TarOptions{})
    if err != nil {
        panic(err)
    }

    return ctx
}

func testImageBuild() {
    ctx := context.Background()
    cli, err := client.NewEnvClient()
    if err != nil {
        log.Fatal(err, " :unable to init client")
    }

    var wg sync.WaitGroup
    for i := 0; i < 3; i++ {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            dockerFile := fmt.Sprintf("%d.Dockerfile", i)
            imageBuildResponse, err := cli.ImageBuild(
                ctx,
                GetContext("."),
                types.ImageBuildOptions{
                    Dockerfile: dockerFile,
                    Tags:       []string{fmt.Sprintf("devbuild_%d", i)},
                })
            if err != nil {
                log.Fatal(err, " :unable to build docker image"+string(1))
            }
            defer imageBuildResponse.Body.Close()
            _, err = io.Copy(os.Stdout, imageBuildResponse.Body)
            if err != nil {
                log.Fatal(err, " :unable to read image build response "+string(1))
            }
        }(i)
    }
    wg.Wait()
}

func main() {
    testImageBuild()
}

GetContext is used to tar the directory path as a context for Docker. testImageBuild spins off three different goroutines to build the three different images.

My question is: When I run this, the output to stdout is always the same and seems deterministic, which makes me think that the images aren't actually been built in parallel. I'm not familiar with how docker build its images, and it seems entirely possible that this approach is simply sending requests to docker server in parallel rather than actually building in parallel. Is this true? If so, how can I build my projects in parallel?

score 1 · Accepted Answer · answered Jun 17 '19 at 05:27

1

If I understand your question correctly, you have a docker-machine on which you want to build the images concurrently using your GO program.

I tried to do the same thing with Dockerfiles which have the same image being built, and per my understanding, all of them were build concurrently.

Here is the go package that I used to replicate the scenario - https://github.com/nihanthd/stackoverflow/tree/master/docker

Now in your case if you were using 3 different docker files, then certainly they would have different build times, that means the output would be seem to be deterministic

answered Jun 17 '19 at 05:27

Nihanth Dara

108
1
5

How were you able to verify they built concurrently? – bli00 Jun 18 '19 at 00:07
By checking the timestamps at which the images were created in the docker-machine. You can ssh into the docker machine and then do `docker image ls -a | grep dev`, as well as the [log line](https://github.com/nihanthd/stackoverflow/blob/master/docker/build.go#L43) in the code I shared should print out the timestamp at which the container was created. – Nihanth Dara Jun 18 '19 at 21:52

Build docker images in parallel using docker Go client

1 Answers1