Standard Go Project Layout with Docker causes build context problem

Question

I'm following two guides which appear to conflict

Go's Standard Go Project Layout recommends a /build directory, the use of which is described as

Packaging and Continuous Integration.

Put your cloud (AMI), container (Docker), OS (deb, rpm, pkg) package configurations and scripts in the /build/package directory.

Following Google Cloud Endpoints Sample for Go using gRPC, I have a Dockerfile which copies application source code and installs any dependencies with go get.

# /build/Dockerfile

FROM golang:alpine

# Alpine Linux package management
RUN apk update
RUN apk add git

COPY ../ /go/src/github.com/username/repository

# Downloads the packages named by the import paths, along with their
# dependencies. It then installs the named packages, like 'go install'.
# ****Don't do this in production! Use vendoring instead.****
RUN go get -v github.com/username/repository/foo-module

# Compiles and installs the packages named by the import paths.
RUN go install github.com/username/repository/foo-module

ENTRYPOINT ["/go/bin/foo-module"]

Following Standard Go Project Layout I place the aforementioned Dockerfile in /build.

Since the Dockerfile is now in the /build directory, I modified the COPY command to copy the parent directory to find the application source code (COPY ../ /go/src/github.com/username/repository).

The Docker build command is not run directly, rather the build is started with Google Cloud Build

cloud-build-local --config=build/cloudbuild.yaml .

The cloudbuild.yaml file is really simple and just kicks off the Docker build. The --file flag points to build/Dockerfile since the Cloud Build command is started from the project root and not from /build)

# /build/cloudbuild.yaml

steps:
    - id: Build
      name: "gcr.io/cloud-builders/docker"

      dir: ${_SOURCE}
      args:
        [
            "build",
            "--file build/Dockerfile",
            ".",
        ]

As expected, this fails

COPY failed: Forbidden path outside the build context: ../ ()

How to include files outside of Docker's build context? suggests using the --file flag, however, this assumes the build is started from the root context. When using Google's Cloud Build, the build is started from cloudbuild.yaml, which also located in /build.

I could just place the Docker file in the root of my go module, but I would like to follow best practices where possible and keep cloudbuild.yaml and Dockerfile in /build.

What is the correct way to achieve this while following the Standard Go Project Layout?

score 1 · Accepted Answer · answered Sep 19 '20 at 18:26

1

That's a rather long question, so lets focus on the problem encountered:

COPY ../ /go/src/github.com/username/repository

which resulted in

COPY failed: Forbidden path outside the build context: ../ ()

You don't include files outside of the context, docker doesn't allow that. Instead you change your context to include the files you need to build your image, and make the paths in your COPY/ADD commands relative to that context. Make sure to reread that, these paths are not relative to the Dockerfile location.

So with a docker build, if you have build/Dockerfile, you would build from the parent directory with:

docker build -f build/Dockerfile .

That trailing dot is the path to the build context which gets sent to the docker engine to perform the build. It doesn't access files directly on the client, which is why you can't include files from outside of the context (plus you don't want malicious Dockerfiles extracting data from the build server).

Then inside the Dockerfile you'd define the paths relative to that context:

COPY . /go/src/github.com/username/repository

And if for some reason you cannot build from that parent directory because of the tooling, then make the context the parent folder with a relative path:

docker build -f Dockerfile ..

answered Sep 19 '20 at 18:26

BMitch

231,797
42
475
450

Thank you. This make sense and when applied to Cloud Build means I simply add `".."` as the final `arg` when running `gcr.io/cloud-builders/docker` from `cloudbuild.yaml`. However this errors `error checking context: 'file ('/proc/6/fd/5') not found or excluded by .dockerignore'`, so something isn't right. My `. dockerignore` doesn't ignore any build files – Jack Sep 19 '20 at 18:58
@Jack I'm guessing you don't want `..` in that builder, just fix the copy line in the Dockerfile to have a `.` and stick with the yaml you posed in the question. – BMitch Sep 19 '20 at 19:00
Without `".."` as the final `arg`, Docker's context will be limited to `/build` and `COPY ../` fails with `COPY failed: Forbidden path outside the build context:`. I guess I misunderstood your answer. – Jack Sep 19 '20 at 19:04
@jack reread the bold part, and fix the copy line in the Dockerfile to have a `.` rather than a `..`. If you're telling it to use `build/Dockerfile` you're not building from within `build` so the assumptions you're making are incorrect. – BMitch Sep 19 '20 at 19:12
This worked (thank you). I think this has more to do with Cloud Build, when running from the root of the project, the trailing `.` means that the build is run in the context of the root dir: `cloud-build-local --config=build/cloudbuild.yaml --dryrun=false .`. So when Docker `build` is run (via `build/cloudbuild.yaml`) there is no need to specify the Docker build context as the parent (`..`), Cloud Build has already done that by the time Docker `build` enters the scene. – Jack Sep 19 '20 at 19:20

Standard Go Project Layout with Docker causes build context problem

1 Answers1