4

In a large C project, I have a top Makefile and many sub-Makefiles in different subdirectories. I need to collect all dependencies of the compilation. For that, I add -MMD to CFLAGS and get a bunch of .d dependency files.

These .d files are scattered in the subdirectories. Also, the dependencies are written sometimes as absolute paths, sometimes as paths relevant to the compilation directory, and sometimes containing symbolic links. I have written a script which finds all .d files, traverses their directories, and resolves all found paths. This works, but with tens of thousands of dependency files this dependency collection lasts about the same time as the compilation! (which is too long to wait :) )

Is there a faster way to get all dependencies in a single file? This is ANSI C, GCC and Linux if that matters. Thanks in advance.

Konstantin Shemyak
  • 2,369
  • 5
  • 21
  • 41

2 Answers2

1

Instead of -MMD, you can use -MM, which sends the dependencies to standard output.

You can then collect all the output to some dependency file in the top level directory with 

gcc -MM ... file.c >>$(top)/all.d

If post processing is the only reason for collecting the output in one file, you can filter the output with a pipe 

gcc -MM ... file.c | sh filter.sh >file.d

and keep the dependency files separate.

If the path to some local include file (defs.h) or the main source is important, you can force gcc to include a path by giving the appropriate -I option, e.g. 

gcc -MM -I$(top)/path/to ... $(top)/path/to/file.c >>$(top)/all.d

or 

gcc -MM -I$(top)/path/to ... $(top)/path/to/file.c | sh filter.sh >file.d

Instead of 

file.o: file.c defs.h

gcc will emit 

file.o: /absolute/path/to/file.c /absolute/path/to/defs.h

This works with relative paths as well, of course.

Olaf Dietsche
  • 72,253
  • 8
  • 102
  • 198
  • But I have many dependencies with identical filenames, such as `dir1/defs.h` and `dir2/defs.h`, included by source files from their corresponding directories by relative path (with `#include "defs.h"`). `gcc -MM ... file.c >>$(top)/all.d` will output just `defs.h`, and I'll not know which of `defs.h` was used. – Konstantin Shemyak May 20 '14 at 04:51
  • How about the second alternative? – Olaf Dietsche May 20 '14 at 05:52
  • The second alternative does not differ much from what I'm doing now: the filter would have to receive the path of the **gcc** being executed in addition to the stdin. Now my script receives the path of the **.d** file from output of **find**. I guess the only thing which can do it "faster" is an option of GCC which would produce **resolved absolute** paths for the dependency files. No such option seems to exist... – Konstantin Shemyak May 20 '14 at 10:02
  • See also question http://stackoverflow.com/q/18136918/1741542 for determining current directory with make. – Olaf Dietsche May 20 '14 at 10:17
  • Thank you for the answer, but I cannot make it work. If my **test.c** file contains just one line, `#include defs.h`, then **gcc -I$PWD -MM test.c** outputs **test.o: test.c defs.h**. I am not getting the absolute path to the dependency. – Konstantin Shemyak May 21 '14 at 09:33
  • You must enclose variables in parenthesis, e.g. `-I$(PWD)`. Without parenthesis, make interprets `-I$PWD` as `$P` and `WD`, which results in `-IWD`. See also [GNU make - Basics of Variable References](http://www.gnu.org/software/make/manual/make.html#Reference) – Olaf Dietsche May 21 '14 at 12:23
  • Olaf, thank you again. This was just a shell command, not used in a Makefile, so it was shell's $PWD (instead of "pwd" in backticks, as backticks are interpreted by StackOverflow markup in the comments). ...And now I understood what you (probably :) ) were telling me: pass the full path to the **.c** file! (full path to -I option is not in fact needed, and it confused me). Then GCC really outputs the full path to the dependency. Seems like not-too-well documented behaviour... Happily accepting the answer. – Konstantin Shemyak May 22 '14 at 05:41
  • And sorry I was wrong about '-I' GCC option: **-I** is also needed in order to get absolute paths before the dependencies, included from the corresponding directories. All clear now. – Konstantin Shemyak May 22 '14 at 10:47
0

You can create the dependency files along with the first compile run.

During the first run, the objects do not exist yet, so the compiler will be invoked anyway. Create empty dependency files first, then update them while compiling.

It should be possible to extend the minimal Makefile for a single-directory C++ project to work with subdirectories.

Community
  • 1
  • 1
Simon Richter
  • 28,572
  • 1
  • 42
  • 64