Change older makefile system to take advantage of parallel compiles

Question

We use Microsoft NMAKE to compile a large number of native C++ and some Intel Fortran files. Typically the makefiles contains lines such as this (for each file):

$(LINKPATH)\olemisc.obj : ole2\olemisc.cpp $(OLEMISC_DEP)
$(CCDEBUG) ole2\olemisc.cpp
$(GDEPS) ole2\olemisc.cpp

OLEMISC_DEP =\
e:\ole2\ifaceole.hpp\
e:\ole2\cpptypes.hpp\
etc.

It works fine, but compiles one file at a time. We would like to take advantage of multi core processors and compile more than one file at a time. I would appreciate some advice about the best way to make that happen, please. Here is what I have so far.

One: GNU make lets you execute parallel jobs using the --jobs=2 option for example and that works fine with GCC (we cant use GCC sadly). But Microsoft's NMAKE does not seem to support such an option. How compatible would the two name programs be, and if we did start using GNU MAKE, can you run two cl.exe processes at the same time? I would expect them to complain about the PDB (debug) file being locked, or does one of the newer cl.exe command line arguments get you around that?

Two: cl.exe has a /MP (build with multiple processes) flag, which lets you compile multiple files at the same time if passed together via the command line, for example:

cl /MP7 a.cpp b.cpp c.cpp d.cpp e.cpp

But using this would require changes to the makefile. Our make files are generated by a our own program from other files, so I can easily change what we put in the makefiles. But how do you combine the dependencies from different cpp files together in the makefile so they get compiled together via one cl.exe call? Each .obj is a different target with a set of commands to make it?

Or do I change the makefile to not call cl.exe, but rather some other little executable that we write, which then collects a series of .cpp files together and shells out to cl.exe passing multiple arguments? That would work and seems doable, but also seems overly complicated and I cant see anyone else doing that.

Am I missing something obvious? There must be a simpler way of accomplishing this?

We do not use Visual Studio or a solution file to do the compiles, because the list of files is extensive, we have a few special items in our makefiles, and theoretically do not want to be overly tied to MS C++ etc.

Answer 1

I thoroughly recommend GNU make on windows. I tend to use cygwin make as the environment it creates tends to be very portable to Unix-like platforms (Mac and Linux for a start). Compiling using the Microsoft toolchain, in parallel and with 100% accurate dependencies and CPU usage works very well. You have other requirements though.

As far as your nmake question goes, look up batch-mode inference rules in the manual. Basically, nmake is able to call the C compiler once, passing it a whole load of C files in one go. Thus you can use the compiler's /MP... type switches.

Parallel compiling built into the compiler? Pah! Horribly broken I say. Here is a skeleton anyway:

OBJECTS = a.obj b.obj c.obj
f.exe: $(OBJECTS)
    link $** -o $@

$(OBJECTS): $$(@R).c

# "The only syntactical difference from the standard inference rule
# is that the batch-mode inference rule is terminated with a double colon (::)."
.c.obj::
    cl -c /MP4 $<

EDIT

If each .obj has its own dependencies (likely!), then you simply add these as separate dependency lines (i.e., they don't have any shell commands attached).

a.obj: b.h c.h ../include/e.hpp
b.obj: b.h ../include/e.hpp
    ∶

Often such boiler plate is generated by another tool and !INCLUDEd into the main makefile. If you are clever, then you can generate these dependencies for free as you compile. (If you go this far, then nmake starts to creak at the seams and you should maybe change to GNU make.)

Answer 2

One further consideration to keep in mind here is this: You basically have to define one batch rule for each path and extension. But if you have two files with the same name in two different source directories with a batch inference rule for both of those directories, the batch rule might not pick the one you want.

Basically the make system knows it needs to make a certain obj file, and as soon as it finds an inference rule that lets it do that, it will use it.

The work around is to not have duplicate named files, and if that cant be avoided, dont use inference or batch rules for those files.

Answer 3

Ok, I spent some time this morning working on this, and thanks to bobbogo, I got it to work. Here are the exact details for anyone else who is considering this:

Old style makefile which compiles one file at a time has tons of this:

$(LINKPATH)\PS_zlib.obj : zlib\PS_zlib.cpp $(PS_ZLIB_DEP)
        $(CC) zlib\PS_zlib.cpp

$(LINKPATH)\ioapi.obj : zlib\minizip\ioapi.c $(IOAPI_DEP)
        $(CC) zlib\minizip\ioapi.c

$(LINKPATH)\iowin32.obj : zlib\minizip\iowin32.c $(IOWIN32_DEP)
        $(CC) zlib\minizip\iowin32.c

Note that each file is compiled one at a time. So now you want to use the fancy Visual Studio 2010 /MP switch "/MP[n] use up to 'n' processes for compilation" to compile multiple files at the same time. How? Your makefile needs to make use of batch inference rules in nmake, as follows:

$(LINKPATH)\PS_zlib.obj : zlib\PS_zlib.cpp $(PS_ZLIB_DEP)

$(LINKPATH)\ioapi.obj : zlib\minizip\ioapi.c $(IOAPI_DEP)

$(LINKPATH)\iowin32.obj : zlib\minizip\iowin32.c $(IOWIN32_DEP)

#Batch inference rule for extension "cpp" and path "zlib":
{zlib}.cpp{$(LINKPATH)}.obj::
        $(CC) $(CCMP) $<

#Batch inference rule for extension "c" and path "zlib\minizip":
{zlib\minizip}.c{$(LINKPATH)}.obj::
        $(CC) $(CCMP) $<

In this case, elsewhere, we have

CCMP = /MP4

Note that nmake inference batch rules do not support wildcards or spaces in the paths. I found some decent nmake documentation somewhere that states that you need to create a separate rule for every extension and source file location, you can not have one rule if the files are in the different locations. Also, files that use #import can not be compiled with /MP.

We have a tool that generates our makefiles, so it now also also generates the batch inference rules.

But it works! The time to compile one large dll went from 12 minutes down to 7 minutes! Woohoo!