makefile - build mulitple files with multiple targets

Question

I have a project with multiple Dockerfiles, each located in a named directory that I use as the Docker image tag. I need to build, test, and push each a docker image for each Dockerfile. What do I need to do to make something such as the following work with GNU Make?

# BUILDS needs to be a list of directories, not a list of Dockerfiles
BUILDS  := $(wildcard */Dockerfile)
VERSION := $(shell git rev-parse --short=12 --verify HEAD)
DOCKER_REPO_URL := quay.io/reponame

define docker_build =
$(1):
    @echo "Building $$@"
    docker build -t $$@ --force-rm $$@
endef

define docker_test =
$(1):
    @echo "Testing $$@"
    docker inspect $$@
    docker run --rm $$@ help
endef

define docker_push =
$(1):
    @echo "Pushing $$@"
    docker tag $$@ $(DOCKER_REPO_URL):$$@-$(VERSION)
    docker push $(DOCKER_REPO_URL):$$@-$(VERSION)
    docker tag $$@ $(DOCKER_REPO_URL):$$@
    docker push $(DOCKER_REPO_URL):$$@
endef

.PHONY: all build test release clean

all: build test release

build: $(BUILDS)
$(foreach build,$(BUILDS),$(eval $(call docker_build,$(build))))

test: $(BUILDS)
$(foreach test,$(BUILDS),$(eval $(call docker_test,$(test))))

release:
$(foreach image,$(BUILDS),$(eval $(call docker_push,$(image))))

I am not familiar with docker. If I have `foo/Dockerfile` and I execute `docker build -t foo --force-rm foo`, is the "docker image" it produces a file? And if so, what is the name of the file and where does it reside? — Beta, Apr 12 '16 at 23:46
Hello Beta, the images (files) are on the docker host and depend on the storage driver used. http://stackoverflow.com/questions/19234831/where-are-docker-images-stored-on-the-host-machine — Tom, Apr 14 '16 at 22:24
Do you mean that in general you don't know where the file will go or what it will be called? In that case an *effective* makefile is easy, but an *efficient* one is not. — Beta, Apr 14 '16 at 23:00
I mean the makefile should not be dependent on a specific docker configuration. The host could be on a remote machine and could utilize several storage drivers. docker build can tag the image with the dir name ' -t ${PWD##*/} '. docker run and push can then ref the dir name too. — Tom, Apr 15 '16 at 07:01

score 6 · Accepted Answer · answered Apr 16 '16 at 01:47

I'm not sure this is what you want, but...

First consider the BUILD variable. If we have three Dockerfiles:

foo/Dockerfile
bar/Dockerfile
baz/Dockerfile

then we want BUILDS to contain foo bar baz

Here are a couple of attempts:

BUILDS := $(wildcard */Dockerfile) # this is foo/Dockerfile bar/Dockerfile baz/Dockerfile

BUILDS := $(dir $(wildcard */Dockerfile)) # this is foo/ bar/ baz/

BUILDS  := $(patsubst %/,%, $(dir $(wildcard */Dockerfile))) # this is foo bar baz

Crude but effective.

Now the rules. Ordinarily the target of a rule is the name of a file which the rule builds. In this case we must break this convention, since we don't know what the name of the image file will be. So if the directory is foo/, we could have a rule called build_foo:

build_foo:
    @echo "Building foo"
    @echo docker build -t foo --force-rm foo

Since we don't want to write a rule for every possible directory, we will use automatic variables and create a pattern rule:

build_%:
    @echo "Building $$@"
    @echo docker build -t $* --force-rm $*

Now "make build_foowill work correctly. And we could write abuild` rule that builds all of them:

build: $(addprefix build_,$(BUILDS))

But this is not quite the right approach. We want to build, then test, then push each image, in that order. So we'd like something like this:

push_foo: test_foo

test_foo: build_foo

We can do this with pattern rules:

test_%: build_%
    ...

push_%: test_%
    ...

release: $(addprefix push_,$(BUILDS))

Now "make release" will do everything. (And if you put release: as the first rule in the makefile, it will be the default rule, and "make" will suffice.)

What an outstanding answer. Thank you. I appreciate the explanations and demonstrating both of the flows. I did implement the recomended flow. — Tom, Apr 16 '16 at 15:58
This solution also allows one off rules such as, 'make build_1.2-node5.10-onbuild' - any possible way to get tab completion? — Tom, Apr 16 '16 at 22:07
@Tom: Tab completion? You mean so that when you edit the makefile, the tabs will be taken care of automatically? It depends on your text editor. — Beta, Apr 17 '16 at 15:13

score 0 · Answer 2 · answered Apr 16 '16 at 07:28

Like @Beta, I don't see why you want to build all the images, then test all the images, then push all the images, as opposed to building, testing and pushing each image; and the latter approach lends itself to a simpler and more normal makefile.

If you have reasons for having to do it the first way, then you'd need a makefile something like this:

# Assuming each subdirectory `foobar` containing a Dockerfile
# is where we `docker build` the image `foobar` 
IMAGES  := $(patsubst %/,%,$(dir $(wildcard */Dockerfile)))
BUILD_TARGS = $(patsubst %,build_%,$(IMAGES))
TEST_TARGS = $(patsubst %,test_%,$(IMAGES))
PUSH_TARGS = $(patsubst %,push_%,$(IMAGES))

VERSION := 1 # $(shell git rev-parse --short=12 --verify HEAD)
DOCKER_REPO_URL := quay.io/reponame

define docker_build =
build_$(1):
    @echo "Building $(1)"
    #docker build -t $(1) --force-rm $(1)
endef

define docker_test =
test_$(1):
    @echo "Testing $(1)"
    #docker inspect $(1)
    #docker run --rm $(1) help
endef

define docker_push =
push_$(1):
    @echo "Pushing $(1)"
    #docker tag $(1) $(DOCKER_REPO_URL):$(1)-$(VERSION)
    #docker push $(DOCKER_REPO_URL):$(1)-$(VERSION)
    #docker tag $$@ $(DOCKER_REPO_URL):$(1)
    #docker push $(DOCKER_REPO_URL):$(1)
endef

.PHONY: all build test release clean $(IMAGES) $(BUILD_TARGS) $(TEST_TARGS) $(PUSH_TARGS)

all: build test release

build: $(BUILD_TARGS)
test: $(TEST_TARGS)
release: $(PUSH_TARGS)

$(foreach image,$(IMAGES),$(eval $(call docker_build,$(image))))
$(foreach image,$(IMAGES),$(eval $(call docker_test,$(image))))
$(foreach image,$(IMAGES),$(eval $(call docker_push,$(image))))

Obviously, uncomment the docker commands to make them run, and restore the correct definition of VERSION.

As it stands it will give you the like of:

$ make
Building foo
#docker build -t foo --force-rm foo
Building bar
#docker build -t bar --force-rm bar
Testing foo
#docker inspect foo
#docker run --rm foo help
Testing bar
#docker inspect bar
#docker run --rm bar help
Pushing foo
#docker tag foo quay.io/reponame:foo-1 
#docker push quay.io/reponame:foo-1 
#docker tag push_foo quay.io/reponame:foo
#docker push quay.io/reponame:foo
Pushing bar
#docker tag bar quay.io/reponame:bar-1 
#docker push quay.io/reponame:bar-1 
#docker tag push_bar quay.io/reponame:bar
#docker push quay.io/reponame:bar

Thank you for the answer Mike. It is correct, but I accepted Beta's because it was first and because I think it goes further to convey an understanding as well — Tom, Apr 16 '16 at 16:02
@Tom I quite agree. You'd do it that way unless there was some unusual reason not. — Mike Kinghan, Apr 16 '16 at 16:43

makefile - build mulitple files with multiple targets

2 Answers2

Linked