156

I'm looking at starting a new project in C++ - just in my own time initially - and I'm investigating the build systems that are available. It would appear that the answer is "Many, and they're all awful".

The features that I specifically need for this are:

  1. C++11 support
  2. Cross platform (Linux as main target, but able to build on at least Windows as well)
  3. Decent unit testing support
  4. Support for multiple modules for separating code out
  5. Support for code generation (Using asn1c or protobuf - not 100% sure yet)
  6. Easy to maintain

Now, I know I can do 1-4 of those using CMake and Autotools easily enough. Probably also with SCons and Waf and the couple of others too. The problem is that I've never worked out how to correctly do code generation using them - that is source files that do not exist until the build process is first run, so source files that the build system must be able to convert into executable code but doesn't actually know about until the build starts... (ASN1C in particular generates dozens of header and source files that must be able to work together, and the actual set of files generates depends on the contents of your asn file) There's also the fact that none of these are especially easy to maintain - CMake and Autotools have their own huge set of scripts that you need to manage for them to work, and Waf and Scons require that anybody working with them has decent knowledge of python (I don't) to work with them...

So - what build systems are recommended for something like this? Or will I be stuck with make files and shell scripts for now?

Graham
  • 4,095
  • 4
  • 29
  • 37
  • 15
    "It would appear that the answer is 'Many, and they're all awful'" is my impression too (with the addition of 'awful from my point of view'; I don't like to generalize too much with terms like these). I actually set up my own for that very reason, and it worked out better than expected since it does what I want, and how I always wanted things to work. For something a little less time consuming, you'll probably have to go through the existing tools and choose one that gives you fewer headaches than the others. – Christian Stieber Aug 18 '12 at 10:01
  • 4
    Try [tup](http://gittup.org/tup/make_vs_tup.html). – Kerrek SB Aug 18 '12 at 10:17
  • see also: http://stackoverflow.com/q/3349956 – ergosys Aug 18 '12 at 22:33
  • 8
    What kind of C++11 support do you expect from a build system? This is something you get from the compiler, the build system doesn't parse or even read the actual source files, just passes them around to whoever needs them, no? – Baruch Sep 03 '13 at 14:49
  • 7
    True, but making it easy to tell the compiler to use C++11 support would be good. g++ needs one flag, clang a different set, msvc apparently doesn't need any and so on. Also, support for detecting what c++11 features are available would be useful as that also differs between compilers... – Graham Sep 04 '13 at 17:17
  • I realize this is an old question, but for generated sources I think a "flag" file can be used. Make the main build depend on the flag file and let the rule that creates the flag file do `generate_files and touch flag`. Or, possible something similar to how dependency files are generated could be done (include a .d for every asn file. Let a pattern rule generate the .d files. The .d files say `binary: asn_gen1 asn_gen2`). I've remember troubles in the past with getting .d files generated on the first run, but I think it was due to some logic error. – thomasa88 Jan 01 '15 at 12:09
  • @KerrekSB as someone who has been using Tup almost exclusively, be careful with it. It messes up symbol loading when you try to debug! – Qix - MONICA WAS MISTREATED Jun 03 '15 at 20:08
  • @Qix: Interesting, thanks. I had mentioned it after I read up on it following a recommendation, but I haven't tried it. Good to know. – Kerrek SB Jun 03 '15 at 20:12
  • @KerrekSB check [this other comment I just made](http://stackoverflow.com/questions/12017580/c-build-systems-what-to-use#comment49325266_27490328) - as with any system, just know what you're getting into :) – Qix - MONICA WAS MISTREATED Jun 03 '15 at 20:13
  • [QBS](https://github.com/qbs/qbs), the one and only non-awful build system out there – sunny moon Jan 22 '19 at 20:22
  • Just use `make` - simple and powerful. – Galik Jul 16 '21 at 16:21

8 Answers8

137

+1 for, "Many, and they're awful."

But, the "richest" and "most-scalable" is probably CMake, which is a Makefile-generator (also generates native MSVC++ *.proj/*.sln). Weird syntax, but once you learn it, it can allow you to nicely generate builds for different platforms. If I "started-fresh", I'd probably use CMake. It should handle your list, although your "code-generation" could take on "a-life-of-its-own" beyond the build system depending on what you want to do. (See below.)

For simple projects, the QMake generator is ok (you don't need to use the Qt libraries to use QMake). But, you're not describing "simple" -- code generation and "extra-phases" means you probably want CMake or something with a rich API for your own extensions, like Scons (or Waf).

We use Scons at work. It produces "bullet-proof-builds", but it's really slow. No other system will be as bullet-proof as Scons. But, it's slow. It is written in Python and we've extended the interface for our "workspace-organization" (where we merely specify module dependencies), and that is part of the Scons design intent (this type of extension through Python). Convenient, but builds are slow. You get bullet-proof builds (any developer box can make the final release), but it's slow. And, it's slow. Don't forget that if you use Scons, though, that it's slow. And, it's slow.

It makes me ill to think that a decade after the Year 2000, we still don't have flying cars. We'll probably have to wait another hundred years or something to get them. And, we will then all probably be flying around in our flying cars that are still being constructed with crappy build systems.

Yes, they are all awful.

[ABOUT CODE GENERATION]

Scons works on "phases", and they are "somewhat-static". It can build code that is generated as part of the build (people are doing this in a couple of different ways), but this has been described as, "something very un-Scons-like".

If it's simple "preprocess some files and generate source files", then no biggie (you have lots of options, and this is why qmake was written -- for the moc preprocessing of *.hpp/*.cpp files).

However, if you are doing this in a "heavy-manner", you're going to need to script your own. For example, we had as-a-part-of-the-build scripts that queried the databases and generated C++ classes to interface between the "layers" (in traditional 3-tier application development). Similarly, we generated server/client source code through IDLs, and embedded version information to permit multiple clients/servers to run simultaneously with different versions (for the same "client" or "server"). Lots of generated source code. We could "pretend" that is "the-build-system", but really, it's a non-trivial-infrastructure for "configuration management", of which part of it is the "build-system". For example, this system had to, "take-down" and "start-up" servers as a part of this process. Similarly, the regression-tests were executed as a part of this process, with heavy "reporting" and "difference-testing" between versions -- all as a part of our "build-scripts".

charley
  • 5,913
  • 1
  • 33
  • 58
  • 50
    I like Scons too - but i think it's slow. – Lothar Aug 18 '12 at 22:26
  • 1
    You can get CMake to handle code generation with a Makefile wrapper to do a an optional second phase when code is generated. I was able to support full dependency tracking, early exit after code generation to trigger a re-cmake, etc. See: http://javaglue.com/javaglue.html#tag:JavaGlue and https://code.google.com/p/javaglue/ – sdw Jun 18 '13 at 02:28
  • SCons & auto-generated code work well together. In our system we auto-generate c++ headers, obj-c files, java files, & can do more. But.. SCons does not "necessarily" support it - nor hinder it. We use a custom builder. It tells SCons "I take list S as source and generate target list T as output". This allows SCons to work out the dependency order. Hence it can support parallel builds as well as the SCons cache. Cons - I think I hear speed (and its not good) but really getting people to understand SCons is the big one. Python is simple, SCons is not. – Shane Gannon Mar 18 '14 at 21:21
  • 4
    Nearly 2 years on, do you still consider SCons to be slow? When I was choosing a build system, I came across [this](http://www.scons.org/wiki/WhySconsIsNotSlow) and it contributed to my decision to go with SCons. – JBentley May 24 '14 at 09:40
  • What about Jenkins? No one mentioned it here.Is it that bad for C++? – Michael IV May 28 '15 at 19:43
  • 1
    @Michael That's a continuous integration tool, not a build system. – Kyle Strand Nov 28 '15 at 18:55
  • 1
    I've heard Scons is slow. What are your thoughts? – Ben Collins Apr 20 '16 at 13:53
  • 4
    @Ben, that's true, but also it's slow. – charley Apr 20 '16 at 14:15
  • 1
    how about bazel now? – ryancheung Nov 01 '16 at 05:19
  • I have only used Scons for small projects up to 250KLoC. I don't remember it being slow - what does it mean "slow"? When does "slow" happen? How does "slow" happen? Is it "slow" after all, or just "slow"? How does it manifest? – Sebastian Mach Jun 28 '17 at 04:52
  • 3
    Anyone looking at this should consider Meson (which uses ninja). – Matthew D. Scholefield Jul 18 '17 at 21:06
  • @MatthewD.Scholefield CMake can use ninja too. It is actualy a shame that it still defaults to gnumake generator. – Slava Dec 06 '17 at 15:26
  • @JBentley In 2017, I tested again Scons for the "Wonderbuild" benchmark that's linked in the link you gave, and nothing changed: it's still as slow as it was 7 years before. I remember from the developer's mailing list that nobody was able to spot any specific place in its source code that was a performance bottleneck; *everything* was kind of slow. So, yes, it's slow, by (lack of proper) design. – Johan Boulé May 06 '18 at 23:32
  • So, why exactly is a C++ build tool written in Python? They couldn't build a C++ build system tool in C++? Wouldn't it be faster? – Aaron Franke Feb 19 '19 at 07:44
  • @AaronFranke, the `build2` toolchain is written in C++: https://build2.org/build2-toolchain/doc/build2-toolchain-intro.xhtml Also, I think `boost.build` is written in C++. – charley Feb 19 '19 at 12:45
  • 5
    :@ CMake is terrible. I'm new to C++ (but 22 years codding) I spent 6 weeks trying to understand it, i just wanted it to download dependencies and compile them (like modern build systems in other languages) and I gave up. Moved over to bazel.io which does this with simple configuration files. I'm still rather shocked that C/C++ is so far behind the 8ball with build tools. No central repo of library descriptions. No standard way for project to describe their build requirements across platform. No standard way to test the code from that description. Man... – CpILL Apr 26 '19 at 08:22
  • @CpILL: You're not looking for a build system, but a package management system. I don't quite see how downloading and installing dependencies is the domain of a build system, especially since a developer usually is not (and indeed should not be!) in the position to install third-party software / system libraries. – DevSolar Jun 26 '19 at 09:18
  • @DevSolar that is exactly my point. Most other package management systems install (i.e. copy from a repo online) to a folder local to the project. Bazel install them to /tmp and then compiles if missing. I think your idea about separating the build system from the package management is dated as you can't build without the dependencies... and now with docker who asks the sys admin if they can install a lib into their container? – CpILL Jun 27 '19 at 05:06
  • @CpILL: That is exactly my point. Confounding a build system with a package manager and a virtualization platform and then blaming the resulting mess on the *language* (or the build system) means you have utterly missed what it's about. For you, a package manager and a virtualization platform is part of a solution you'd like to see. For me, it would be utterly superfluous and in the way. So you want to build C++ *with* CMake *with* a package manager *and* Docker, I want to build C++ with CMake and pass on the rest. A system that tries to be a jack of all trades will be really good at nothing. – DevSolar Jun 27 '19 at 08:21
  • @DevSolar nobody is talking about virtualization platforms? I just suggested that build system and package managers _are_ always needed together (unless the project is trivial and has no dependencies (but even then other projects need to know how to compile them). I mean when can you compile a project without its dependencies? – CpILL Aug 09 '19 at 01:39
  • We can also try xmake, it is based on lua, but it is fast and easy. and it has a builtin package manager. – ruki Oct 12 '21 at 05:29
  • Regarding QMake, it looks like it's being deprecated moving forward. They want people to swap over to cmake: https://www.qt.io/blog/2019/08/07/technical-vision-qt-6 – Tyler Shellberg Jan 31 '23 at 03:21
35

You can use Gradle now: https://docs.gradle.org/current/userguide/native_software.html

This seems to have matured quite a bit in the years since I originally posted this. The page saying that the project is "incubating" has disappeared, but I can't find any official announcement removing this status.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Nate Glenn
  • 6,455
  • 8
  • 52
  • 95
  • I would agree with Gradle. Gradle is designed to be scalable. However, it depends on the plugin implementation how fast it is. There is also some overhead for gradle itself. Also be aware, that some plugins might need to be customised for your use cases. – JE42 Sep 13 '14 at 08:47
  • Interesting to see gradle's interest in supporting C++. Hope they produces a nice and solid building system for c++ projects that all of us are missing. – hbobenicio Oct 23 '14 at 16:56
  • @squid thanks, updated. – Nate Glenn Aug 06 '15 at 02:09
  • 1
    Gradle is powerful yet simple. No weird syntax, a single gradle.build file is often enough to build an entire project with multiple executable outputs (main, tests, etc). No file dumping accross all source directories. Super friendly for versionning. – Overdrivr Jan 12 '16 at 13:55
  • In my experience with Android, Gradle is slower than Scons. Maybe that is because of Android itself, but IntelliJ's native builds always seemed much faster. I would be wary at least. – Timmmm Jan 16 '17 at 09:35
  • 4
    I've spent the past two days doing battle with Gradle, just trying to create a "hello world" library, app, and gtests, and I can't say I can recommend it for a new project. The tools may all be there, but the documentation is not (giving the benefit of the doubt). Maybe those of you who find it viable could point to your favorite resources? – jwm May 03 '18 at 23:09
16

I found these, I have not personally used all of them yet:

Ninja, a small build system focused on speed. Google now uses Ninja to build Android instead of Make: link.

Shake, a powerful and fast build system.

Tup, a high performance build system. Algorithmic based design. Analysis of Tup.

All are now cross-platform and support Windows. I'm not yet sure about the rest of your requirements as, again, I have yet to test them myself. They are being used in commercial development, CIG picked Ninja up. I've used and love ease and speed of Ninja with a project generator. First two are akin to Scons, Ant, etc.

leetNightshade
  • 2,673
  • 2
  • 36
  • 47
  • 1
    Tup will mess up symbol lookups, doesn't play nicely with any other build system (at all, really), doesn't play nicely with random outputs (such as classes generated by `javac` with inner classes, which are separated into `class$1.class` files), is poorly written and uses system hacks to achieve what it does. It's great for small systems; unmaintainable for larger projects. – Qix - MONICA WAS MISTREATED Jun 03 '15 at 20:12
  • @Qix: Can you not keep the output separate from your repository? – Kerrek SB Jun 03 '15 at 20:17
  • 2
    @KerrekSB You can, but that's not the problem with symbol lookup. Tup uses [FUSE](http://fuse.sourceforge.net/), and mounts its own middleware into `.tup/mnt`. It then runs all of the build programs in a folder (i.e. `.tup/mnt/@tupjob-XXXXX`) as the working directory in order to monitor reads/writes, enforcing the build configuration. It works well, unless the path is stored absolutely (i.e. with symbols). When you compile a binary, the symbol path is stored in the binary itself. That means when GDB tries to load the symbols, it looks for that `tupjob` path, which doesn't exist causing errors. – Qix - MONICA WAS MISTREATED Jun 03 '15 at 20:20
  • 1
    Both Tup and Ninja are very low-level tools, as low or ever lower than Make. They're not tailored to C++. I wouldn't even call these tools *build systems* on their own because they are missing a lot of important higher-level features that true build systems offer to deal with complex real world projects. Some of these systems may use Ninja as a back-end. – Johan Boulé May 06 '18 at 23:43
12

Google build system is a good alternative: http://bazel.io/

Mike B
  • 1,522
  • 1
  • 14
  • 24
  • 3
    "Cross platform (Linux as main target, but able to build on at least Windows as well)". Bazel's FAQ says "We are currently actively working on improving Windows support, but it's still ways from being usable." – Michael Mrozek Apr 05 '16 at 03:05
  • 4
    I am not sure whether it is the system itself or the guy who set up the project, but I had major pain in the back using it @ work. It is very unflexible (in the sense that it only supports one way of doing stuff, often suboptimal, if not creepy), not powerful enough, poorly documented (aside basic cases), has little community so do not expect stackoverflow to come save you, quite a dependency by itself, do not play well with most IDEs on most systems, etc, etc, etc. So unless you are a fan of google - stay away from it (btw there is a facebook version of this, called Buck - for facebook fans) – Slava Dec 06 '17 at 15:22
  • Hey, I found it really _easy_ to use after spending 6 weeks trying to get CMake working. I wanted it to download 3rd party dependencies and compile them (as i was used to nice build tools from python and JavaScript). This made it easy but has the draw back that you might have to write your own bazel files for 3rd parties that don't support it. They need a central repo to store these in i'ze reckon. – CpILL Apr 26 '19 at 08:13
  • We use Bazel at work now for cross compiling for different MCUs and We also use it to cross compile for Embedded Linux. It integrates really well with VS Code by running bazel in dev containers. – Michael Stumpf Feb 10 '23 at 20:14
10

Scons is very friendly and flexible system, but you right, Lothar, it really slow.

But there is a way to increase the performance of programs written in Python. This use of the JIT. Of all known projects, PyPy is a very powerful, fast-growing and motivated JIT-backed - Python 2.7 implementation. PyPy compatibility with Python 2.7 is simply amazing. However, Scons declared as unsupported project on the PyPy compatibility wiki. Waf, on the other hand, modeled as python-based autotools sucessor, is fully supported by PyPy infrastructure. In my projects the speed of the assembly has increased 5-7 times in the transition to PyPy. You can see the performance reports from PyPy.

For modern and relatively fast build system Waf is good choice.

aegor
  • 109
  • 1
  • 6
  • It's worth noting that scons is now marked as "Compatible" on the linked wiki page, so apparently it now works wit PyPy. – Fake Name Jan 05 '16 at 18:49
6

I used SCons and am impressed by this build system. SCons is extensible by python and python itself - it's great, because Python has all that you need, just code the logic, all the low-level functionality is already implemented in SCons and Python and is crossplatform. If have good programming skills then your build scripts will be looking perfect and easy.

Make, CMake and similar build systems seems as trash of macroses. Waf is SCons analog. I'm trying Waf but SCons will be more friendly and so I stayed with SCons.

By crowd opinion SCons is too slow, but in the middle of a project I didn't see any difference between make and SCons by build speed. Instead, SCons has well worked with parallel builds, while make has big troubles with it.

Also, SCons allows you to get - configure, build, deploy, generate configuration from templates, run tests and do any other task that can be done can coding with python and SCons - all in one. That is a very big advantage.

For a simple project CMake is also a good choice.

shuttle87
  • 15,466
  • 11
  • 77
  • 106
Torsten
  • 21,726
  • 5
  • 24
  • 31
  • 4
    My problem here is that I'm spoilt. I'm a Java developer by profession, and tools like Gradle in the Java world are the kind of tools that I'd really like to have for C++ development. I can set up a gradle project, with no external dependencies, using a single line build file. That's it. Multi-module projects that have numerous dependencies are still easy to do as well, and actually weigh in at a lot less configuration than even a simple C++ project... – Graham Aug 18 '12 at 14:36
  • I had problems building other packages which use Scons because it does not abstract system-configuration flags like -I include dirs and -L library dirs. It won't take CFLAGS, you often have to modify each scons script to get it to look in the right place. – ACyclic Sep 21 '16 at 16:11
4

just to add my cents: premake

http://industriousone.com/premake

there is also a dedicated webpage on the wiki.

user827992
  • 1,743
  • 13
  • 25
  • 2
    Unfortunately Premake doesn't support code generation per OP's requirements. And I wouldn't call its support for unit testing "decent", although there's a few things you can do. – ergosys Aug 18 '12 at 22:59
  • @ergosys i notice that ant is not mentioned, `ant` also supports C/C++, see if this is good for you, it's the last name i have, i just use Makefiles and Cmake for this. – user827992 Aug 18 '12 at 23:06
1

You can use Ceedling. Note, however it only supports C at the moment and it is tightly coupled to the author's Unity and CMock testing frameworks.

It can be forked and modified to work with a C++ compiler and unit testing / mocking framework fairly easily.

Also Tup is a worthy mention. It is extremely fast but it knows nothing about testing frameworks etc. which means you'll have to write your own build system using Tup. If you plan on doing TDD, Tup is probably the way to go.

thegreendroid
  • 3,239
  • 6
  • 31
  • 40