0

I am trying to build 3 apps that rely on the same dependencies in parallel inside of a single Dockerfile. Using the answer I found here I got it to the point where it pretends to run in parallel, but it is not.

I am aware this is not the best practice way to accomplish this goal, but I need 3 separate applications built with only 1 Dockerfile. If you have a solution that either makes Docker run in parallel, or an alternate solution that satisfies the above requirements, I would love to hear it!

Overall goal: Speed up the build process when the three apps are deployed.

Why: As these apps get more complicated, the build time will go up.

My Code

Here is a clip of my Dockerfile:

FROM golang:1.19-alpine3.17 AS base

RUN apk add bash ca-certificates git gcc g++ libc-dev make

COPY go.mod .
COPY go.sum .

RUN go mod download

FROM base AS builder1
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app1 ./dest/app1

FROM base AS builder2
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app2 ./dest/app2

FROM base AS builder3
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app3 ./dest/app3

FROM builder1 AS finalbuilder
COPY --from=builder1 path/to/app1 /bin/app1
COPY --from=builder2 path/to/app2 /bin/app2
COPY --from=builder3 path/to/app3 /bin/app3

I build this Dockerfile using DOCKER_BUILDKIT=1 docker build --network=host --target=finalbuilder -t $(DOCKER_BUILDER_IMAGE) -f path/Dockerfile .

Result

It runs successfully, and pretends to build all 3 apps at the same time. (Timer counts up in console on the three builds at the same time) However, the time it takes to do so gives it away. Each app by itself takes roughly 30 seconds to build. When using this method described above, it takes roughly 90 seconds to build all 3. Docker is clearly still building them in order, while pretending not to.

NAFB
  • 13
  • 4
  • 1
    From the answer you linked: *If BuildKit sees that a stage depends on other stages which do not depend on each other, then it will run those stages in parallel (**assuming there are enough threads/CPUs available**)*. Emphasis mine. – Daniel Mann Jun 07 '23 at 22:43
  • Maybe I am missing the way to specify threads available? I saw a few flag options for the docker build command, like --cpuset-cpus, but it did not seem to impact anything. – NAFB Jun 09 '23 at 13:55
  • On further examination, I think the problem is my local computer, as you are suggesting. Thank you. – NAFB Jun 09 '23 at 14:03

1 Answers1

0

Why to not run three simple docker contrainers?

I need 3 separate applications built with only 1 Dockerfile

So it still could be one dockerfile with different params, so, you could just create 3 containers :) Another way - is to run a simple script that will live inside container and will execute commands in a different threads. OR use some bash from here. The idea is that you could run in parallel three build commands from one builder, and RUN looks not necessary here.

bkatrenko
  • 16
  • 1
  • `date sleep 10 & # CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app1 ./dest/app1 sleep 10 & # RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app2 ./dest/app2 sleep 10 # RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo -o path/app3 ./dest/app3 date ` will execute only 10 secs, so you could just substitute sleeps for go build commands – bkatrenko Jun 08 '23 at 13:42
  • You are right, that did run in 10 seconds, using 3 sleeps. But replacing with builds is still 90, meaning the problem is probably my local computer's processing power. I should have tried the sleeps first. Thank you. – NAFB Jun 09 '23 at 14:01