27

A colleague recently revealed to me that a single source file of ours includes over 3,400 headers during compile time. We have over 1,000 translation units that get compiled in a build, resulting in a huge performance penalty over headers that surely aren't all used.

Are there any static analysis tools that would be able to shed light on the trees in such a forest, specifically giving us the ability to decide which ones we should work on paring out?

UPDATE

Found some interesting information on the cost of including a header file (and the types of include guards to optimize its inclusion) here, originating from this question.

Community
  • 1
  • 1
fbrereto
  • 35,429
  • 19
  • 126
  • 178
  • What platform? gcc has options that can help with this (if nobody suggests a better idea) – Nemo Jul 13 '11 at 20:59
  • Possible duplicate of http://stackoverflow.com/questions/42308/tool-to-track-include-dependencies – jfritz42 Sep 30 '15 at 16:17
  • @fbrereto: (A few years later...) I'm working on a type-inference for C and one of its intended uses is that of removing all #includes, letting the engine infer only actually used types, and still get the analysis result with same precision. I'm looking for real-world cases for a paper. Please, reach-me out directly if you'd like to talk about this. The engine has an online interface: http://cuda.dcc.ufmg.br/psyche-c/ – Leandro T. C. Melo Jun 21 '17 at 14:31

8 Answers8

26

The output of gcc -w -H <file> might be useful (If you parse it and put some counts in) the -w is there to suppress all warnings, which might be awkward to deal with.

From the gcc docs:

-H

Print the name of each header file used, in addition to other normal activities. Each name is indented to show how deep in the #include stack it is. Precompiled header files are also printed, even if they are found to be invalid; an invalid precompiled header file is printed with ...x and a valid one with ...!.

The output looks like this:

. /usr/include/unistd.h
.. /usr/include/features.h
... /usr/include/bits/predefs.h
... /usr/include/sys/cdefs.h
.... /usr/include/bits/wordsize.h
... /usr/include/gnu/stubs.h
.... /usr/include/bits/wordsize.h
.... /usr/include/gnu/stubs-64.h
.. /usr/include/bits/posix_opt.h
.. /usr/include/bits/environments.h
... /usr/include/bits/wordsize.h
.. /usr/include/bits/types.h
... /usr/include/bits/wordsize.h
... /usr/include/bits/typesizes.h
.. /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/include/stddef.h
.. /usr/include/bits/confname.h
.. /usr/include/getopt.h
. /usr/include/stdio.h
.. /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/include/stddef.h
.. /usr/include/libio.h
... /usr/include/_G_config.h
.... /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/include/stddef.h
.... /usr/include/wchar.h
... /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/include/stdarg.h
.. /usr/include/bits/stdio_lim.h
.. /usr/include/bits/sys_errlist.h
Multiple include guards may be useful for:
/usr/include/bits/confname.h
/usr/include/bits/environments.h
/usr/include/bits/predefs.h
/usr/include/bits/stdio_lim.h
/usr/include/bits/sys_errlist.h
/usr/include/bits/typesizes.h
/usr/include/gnu/stubs-64.h
/usr/include/gnu/stubs.h
/usr/include/wchar.h
Spudd86
  • 2,986
  • 22
  • 20
  • does there exist something similar for devenv/msvs? – andreas Feb 11 '12 at 14:31
  • @user375251 I don't know, don't use the MS toolchain for anything, check MSDN – Spudd86 Feb 13 '12 at 20:37
  • @Spudd86 can I get such list of includes for every file in project without manually type every filename with -H option? – Alecs Mar 21 '13 at 15:26
  • @Alecs maybe you could muck with your `Makefile` so that you could run a full build with that flag added... or add a second command to your build rules that runs gcc with that flag and redirect it's output to a file or something. – Spudd86 Jul 30 '13 at 19:23
3

If you are using gcc/g++, the -M or -MM option will output a line with the information you seek. (The former will include system headers while the latter will not. There are other variants; see the manual.)

$ gcc -M -c foo.c
foo.o: foo.c /usr/include/stdint.h /usr/include/features.h \
  /usr/include/sys/cdefs.h /usr/include/bits/wordsize.h \
  /usr/include/gnu/stubs.h /usr/include/gnu/stubs-64.h \
  /usr/include/bits/wchar.h

You would need to remove the foo.o: foo.c at the beginning, but the rest is a list of all headers that the file depends on, so it would not be too hard to write a script to gather these and summarize them.

Of course this suggestion is only useful on Unix and only if nobody else has a better idea. :-)

Nemo
  • 70,042
  • 10
  • 116
  • 153
3

a few things-

  • use "preprocess only" to look at your preprocessor output. gcc -E option, other compilers have the function too

  • use precompiled headers.

  • gcc has -verbose and --trace options which also display the full include tree, MSVC has the /showIncludes option found under Advanced C++ property page

Also, Displaying the #include hierarchy for a C++ file in Visual Studio

Community
  • 1
  • 1
kert
  • 2,161
  • 21
  • 22
2

"Large Scale C++ Software Design" by John Lakos had tools that extracted the compile-time dependencies among source files.

Unfortunately, their repository on Addison-Wesley's site is gone (along with AW's site itself), but I found a tarball here: http://prdownloads.sourceforge.net/introspector/LSC-rpkg-0.1.tgz?download

I found it useful several jobs ago, and it has the virtue of being free.

BTW, if you haven't read Lakos's book, it sounds like your project would benefit. (The current edition is a bit dated, but I hear that Lakos has another book coming out in 2012.)

Gnawme
  • 2,321
  • 1
  • 15
  • 21
1

GCC has a -M flag that will output a list of dependencies for a given source file. You could use that information to figure out which of your files have the most dependencies, which files are most depended on, etc.

Check out the man page for more information. There are several variants of -M.

Carl Norum
  • 219,201
  • 40
  • 422
  • 469
1

Personally I don't know if there is a tool that will say "Remove this file". It's really a complex matter that depends on a lot of things. Looking at a tree of include statements is surely going to drive you nuts.... It would drive me crazy, as well as ruin my eyes. There are better ways to do things to reduce your compile times.

  1. De-inline your class methods.
  2. After deinlining them, re-examine your include statements and attempt to remove them. Usually helpful to delete them, and start over.
  3. Prefer to use forward declarations are much as possible. If you de-inline methods in your header files you can do this alot.
  4. Break up large header files into smaller files. If a class in a file is used more often than most, then put it in a header file all by itself.
  5. 1000 translational units is not very much actually. We have between 10-20 thousand. :)
  6. Get Incredibuild if your compile times are still too long.
C.J.
  • 15,637
  • 9
  • 61
  • 77
0

I heard there are some tools do it, but I don't use them.

I created some tool https://sourceforge.net/p/headerfinder may be this is useful. Unfortunately it is "HOME MADE" tool with following issues,

  • Developed in Vb.Net
  • Source code need to compiled
  • Very slow and consumes memory.
  • No help available.
oz123
  • 27,559
  • 27
  • 125
  • 187
Abhay Joshi
  • 161
  • 1
  • 6
0

GCC Has a flag (-save-temps) with which you can save intermediate files. This includes .ii files, which are the results of the preprocessor (so before compilation). You can write a script to parse this and determine the weight/cost/size of what is included, as well as the dependency tree.

I wrote a Python script to do just this (publicly available here: https://gitlab.com/p_b_omta/gcc-include-analyzer).

Paul Omta
  • 1,703
  • 1
  • 12
  • 17