2

The question is already answered on file-level. But I have a bigger project which has quite a lot of inter-project-dependencies (caused by DBus headers, which were generated dynamically).

I ve created the following example (example files as ZIP - the real project is much more complex).

The top-level Makefile is the following:

sub-%:
    $(MAKE) -C $(patsubst sub-%,%,$@)

default:
    $(MAKE) -j12 sub-p1 sub-p2 sub-p3

The Makefile of the sub-project look like this (p1, p2 and p3):

all: p1

../lib/lib.a:
    $(MAKE) -C ../lib lib.a

p1: ../lib/lib.a
    cp -f ../lib/lib.a p1

And the Makefile of the lib looks like this:

lib.a:
    sleep 2
    date > $@
    echo Done with building $@

THE PROBLEM: The library is built for each p*-project separately in parallel - in this example it's not a problem, but in our case it causes unsolvable problems.

When I call make on the top-level, I get the following output:

$ make
make -j12 sub-p1 sub-p2 sub-p3
make[1]: Entering directory '/home/kkr/tmp/parallelmake'
make -C p1
make -C p2
make -C p3
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p1'
make -C ../lib lib.a
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p2'
make -C ../lib lib.a
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p3'
make -C ../lib lib.a
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
sleep 2
sleep 2
sleep 2
date > lib.a
date > lib.a
date > lib.a
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
cp -f ../lib/lib.a p3
cp -f ../lib/lib.a p1
cp -f ../lib/lib.a p2
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p3'
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p1'
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p2'
make[1]: Leaving directory '/home/kkr/tmp/parallelmake'

QUESTION: Is it possible to synchronize the sub-projects somehow automatically?

Since the real project has 13 sub-projects - with most of them having inter-project-dependencies - a manual synchronization would be quite difficult.

Charly
  • 1,270
  • 19
  • 42

1 Answers1

0

There is clearly no way for a recursive make system to "automatically" synchronize this; they are completely separate processes and have no way to tell each other that they are running a given target.

You have two choices. The first, which is best but may be a lot of work, is to rework your build system so it's non-recursive. If you have a single make invocation which builds the entire system then there is only one instance of make, and it will globally coordinate the creation of all targets. You won't need the $(MAKE) -C ../lib lib.a rules at all.

If that's not feasible, then you need to rewrite your top-level makefile so that you enforce an ordering such that you never start making two directories that depend on the same external target, at the same time. For example instead of the top-level makefile you show above, write it like this:

default: p1 p2 p3

p%: ; $(MAKE) -C $@

.PHONY: default p1 p2 p3

Now suppose you want to ensure that p1 runs before p2 and p3 because you want p1 to build all the shared targets; then you'd add a rule like this to the top-level makefile:

p2 p3: p1

And you can add other orderings as needed.

MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • I spent hours trying to get this to work with parallel make. Are you completely sure it works? What does the link rule that depends upon the subproject libraries look like? I can only get it to relink every time or not even link after building subprojects. My Makefile is perfect without `-j`. – doug65536 Nov 19 '16 at 05:05
  • This example doesn't provide enough detail to answer your question. It's an abstract high-level design. And, SO comments are not sufficient for these explanations. If you have a detailed question I recommend you file a new question with more specifics. – MadScientist Nov 19 '16 at 14:03
  • But yes, the design here works. If you need to provide another target at the root level that wraps up all the work done at the individual directory levels, that's an entirely different thing which is not discussed or described here because that's not what this question asked about. – MadScientist Nov 19 '16 at 14:05
  • I figured it would be hard to answer as a comment with no code to look at, I'll try to get an SSCCE put together and ask a proper question. Thanks. – doug65536 Nov 19 '16 at 15:18