133

Somehow I am totally confused by how CMake works. Every time I think that I am getting closer to understanding how CMake is meant to be written, it vanishes in the next example I read. All I want to know is, how should I structure my project, so that my CMake requires the least amount of maintainance in the future. For example, I don't want to update my CMakeList.txt when I am adding a new folder in my src tree, that works exactly like all other src folders.

This is how I imagine my project's structure, but please this is only an example. If the recommended way differs, please tell me, and tell me how to do it.

myProject
    src/
        module1/
            module1.h
            module1.cpp
        module2/
            [...]
        main.cpp
    test/
        test1.cpp
    resources/
        file.png
    bin
        [execute cmake ..]

By the way, it is important that my program knows where the resources are. I would like to know the recommended way of managing resources. I do not want to access my resources with "../resources/file.png"

starball
  • 20,030
  • 7
  • 43
  • 238
Arne
  • 7,921
  • 9
  • 48
  • 66
  • 2
    `For example I don't want to update my CMakeList.txt when I am adding a new folder in my src tree` can you give an example of IDE which collects sources automatically? –  Jan 17 '14 at 07:07
  • 9
    no ide's normally don't collect sources automatically, because they don't need to. When I add a new file or folder, I do it within the ide, and the project is updated. A build system on the other side does not notice when I change some files, so it is a desired behavior that it collects all source files automatically – Arne Jan 17 '14 at 16:26

3 Answers3

104

After some research, I have now my own version of the most simple but complete CMake example. Here it is, and it tries to cover most of the basics, including resources and packaging.

One thing it does non-standard is resource handling. By default CMake wants to put them in /usr/share/, /usr/local/share/ and something equivalent on Windows. I wanted to have a simple zip/tar.gz file that you can extract anywhere and run. Therefore resources are loaded relative to the executable.

The basic rule to understand CMake commands is the following syntax: <function-name>(<arg1> [<arg2> ...]) without comma or semicolon. Each argument is a string. foobar(3.0) and foobar("3.0") is the same. You can set lists/variables with set(args arg1 arg2). With this variable set foobar(${args}) and foobar(arg1 arg2) are effectively the same. A nonexistent variable is equivalent to an empty list. A list is internally just a string with semicolons to separate the elements. Therefore a list with just one element is by definition just that element, no boxing takes place.

Variables are global. Built-in functions offer some form of named arguments by the fact that they expect some ids, like PUBLIC or DESTINATION, in their argument list, to group the arguments. But that's not a language feature; those ids are also just strings, and parsed by the function implementation.

You can clone everything from GitHub.

cmake_minimum_required(VERSION 3.0)
project(example_project)

###############################################################################
## file globbing ##############################################################
###############################################################################

# these instructions search the directory tree when CMake is
# invoked and put all files that match the pattern in the variables
# `sources` and `data`
file(GLOB_RECURSE sources      src/main/*.cpp src/main/*.h)
file(GLOB_RECURSE sources_test src/test/*.cpp)
file(GLOB_RECURSE data resources/*)
# you can use set(sources src/main.cpp) etc if you don't want to
# use globbing to find files automatically

###############################################################################
## target definitions #########################################################
###############################################################################

# add the data to the target, so it becomes visible in some IDE
add_executable(example ${sources} ${data})

# just for example add some compiler flags
target_compile_options(example PUBLIC -std=c++1y -Wall -Wfloat-conversion)

# this lets me include files relative to the root source directory with a <> pair
target_include_directories(example PUBLIC src/main)

# this copies all resource files in the build directory
# we need this, because we want to work with paths relative to the executable
file(COPY ${data} DESTINATION resources)

###############################################################################
## dependencies ###############################################################
###############################################################################

# this defines the variables Boost_LIBRARIES that contain all library names
# that we need to link to
find_package(Boost 1.36.0 COMPONENTS filesystem system REQUIRED)

target_link_libraries(example PUBLIC
  ${Boost_LIBRARIES}
  # here you can add any library dependencies
)

###############################################################################
## testing ####################################################################
###############################################################################

# this is for our testing framework
# we don't add REQUIRED because it's just for testing
find_package(GTest)

if(GTEST_FOUND)
  add_executable(unit_tests ${sources_test} ${sources})

  # we add this define to prevent collision with the main
  # this might be better solved by not adding the source with the main to the
  # testing target
  target_compile_definitions(unit_tests PUBLIC UNIT_TESTS)

  # this allows us to use our executable as a link library
  # therefore we can inherit all compiler options and library dependencies
  set_target_properties(example PROPERTIES ENABLE_EXPORTS on)

  target_link_libraries(unit_tests PUBLIC
    ${GTEST_BOTH_LIBRARIES}
    example
  )

  target_include_directories(unit_tests PUBLIC
    ${GTEST_INCLUDE_DIRS} # doesn't do anything on Linux
  )
endif()

###############################################################################
## packaging ##################################################################
###############################################################################

# all install commands get the same destination. this allows us to use paths
# relative to the executable.
install(TARGETS example DESTINATION example_destination)
# this is basically a repeat of the file copy instruction that copies the
# resources in the build directory, but here we tell CMake that we want it
# in the package
install(DIRECTORY resources DESTINATION example_destination)

# now comes everything we need, to create a package
# there are a lot more variables you can set, and some
# you need to set for some package types, but we want to
# be minimal here
set(CPACK_PACKAGE_NAME "MyExample")
set(CPACK_PACKAGE_VERSION "1.0.0")

# we don't want to split our program up into several things
set(CPACK_MONOLITHIC_INSTALL 1)

# This must be last
include(CPack)

Note: While the above usage of file(GLOB) is appropriate here, where the question specifically asks for a technique to minimize the frequency of edits to CMakeLists.txt files with the addition of new source files, this technique is discouraged in the official documentation, and in the answers to these dedicated questions: #1, #2.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Arne
  • 7,921
  • 9
  • 48
  • 66
  • How does this solve your self-imposed requirement of automatically adding a new module? – André Dec 01 '15 at 10:14
  • 4
    @SteveLorimer I also disagree with the pattern to put one CMakeLists.txt in every directory of the projects, it just scatters the configuration the project everywhere, I think one file to do it all should be enough, otherwise you loose the overview, of what is actually done in the build process. That doesn't mean there can't be subdirectories with their own CMakeLists.txt, I just think it should be an exception. – Arne Apr 06 '16 at 22:41
  • @Arne Thanks for the write-up! But Coverage and Valgrind cmake instructions are missing with respect to GTest. – uss Jul 17 '17 at 14:37
  • @sree Thanks for you feedback. The reason those instructions are missing for GTest are simply I did not think about them, and I don't really use gtest anymore so I don't know. The question would be, is it straight forward to add coverage and valgrind instructions from now on and adding it to this minimal example would just clutter it, or would it really be an improvement? Pull requests to the gihub repository are welcome. – Arne Jul 18 '17 at 12:08
  • 2
    I also would like to state that, for the readers of the project, having a CMakeLists of 2000 lines where 1800 are just references to source files isn't exactly a pleasant reading. – JoaoBapt Sep 12 '20 at 14:56
  • This might be a dumb question. But where does the "enable_testing()" and "add subdirectory(src/test)" go? preferably in a place where it works with visual studio test explorer. – Jelle Bleeker Nov 15 '22 at 13:20
  • @JelleBleeker I don't use `enable_testing()` nor do I actually understand what it is supposed to do. This CMake file just builds the test cases, executing them is a different story. Maybe there might be a better option to add tests in an IDE aware scheme. Regarding `add_subdirectory` I completely avoided the usage of that function intentionally. `add_subdirectory` will look for a CMakeLists.txt file in that subdirectory, spreading cmake logic around in the project. I want all CMake logic to be contained in a single file. Therefore no `add_subdirectory`. – Arne Nov 24 '22 at 20:45
43

The most basic but complete example can be found in the CMake tutorial:

cmake_minimum_required (VERSION 2.6)
project (Tutorial)
add_executable(Tutorial tutorial.cxx)

For your project example you may have:

cmake_minimum_required (VERSION 2.6)
project (MyProject)
add_executable(myexec src/module1/module1.cpp src/module2/module2.cpp src/main.cpp)
add_executable(mytest test1.cpp)

For your additional question, one way to go is again in the tutorial: create a configurable header file that you include in your code. For this, make a file configuration.h.in with the following contents:

#define RESOURCES_PATH "@RESOURCES_PATH@"

Then in your CMakeLists.txt add:

set(RESOURCES_PATH "${PROJECT_SOURCE_DIR}/resources/")
# configure a header file to pass some of the CMake settings
# to the source code
configure_file (
  "${PROJECT_SOURCE_DIR}/configuration.h.in"
  "${PROJECT_BINARY_DIR}/configuration.h"
)

# add the binary tree to the search path for include files
# so that we will find TutorialConfig.h
include_directories("${PROJECT_BINARY_DIR}")

Finally, where you need the path in your code, you can do:

#include "configuration.h"

...

string resourcePath = string(RESOURCE_PATH) + "file.png";
sgvd
  • 3,819
  • 18
  • 31
  • thank you very much, especially for the RESOURCE_PATH, somehow I did not get that the configure_file is what I was looking for. But you added all files from the project manually, is there a better way way to simply define a pattern in which all files are added from the src tree? – Arne Jan 16 '14 at 16:01
  • See Dieter's answer, but also my comments on why you shouldn't use it. If you really want to automate it, a better approach may be to write a script that you can run to regenerate the list of source files (or use a cmake aware IDE that does this for you; I am not familiar with any). – sgvd Jan 16 '14 at 16:37
  • 4
    @sgvd `string resourcePath = string(RESOURCE_PATH) + "file.png"` IMHO it's a bad idea to hardcode **absolute** path to source directory. What if you need to install your project? –  Jan 17 '14 at 07:13
  • The point is that it is not hard coded as such, it is determined at configuration time. You would indeed have to do something a bit smarter to set `RESOURCE_PATH` in your `CMakeLists.txt`, perhaps using `CMAKE_INSTALL_PREFIX` if building a Release version, or add the install path as another configured definition and test both at run time. But it is not hardcoded as in that you have to actually change your code when moving things arounf. – sgvd Jan 17 '14 at 09:57
  • 2
    I know automatically gathering sources sounds nice, but it can lead to all sorts of complications. See this question from a while ago for a brief discussion: http://stackoverflow.com/q/10914607/1401351. – Peter Jan 17 '14 at 14:42
  • I've summed it up and not gathering automatically all sources leads to more complications, for example on cpp file is missing in the cmake, but it is clearly in the project folder which leads in the end to undefined references that you have to track. – Arne Jan 17 '14 at 18:10
  • 2
    You get exactly the same error if you don't run cmake; adding files by hand takes one second one time, running cmake at every compile takes one second every time; you actually break a feature of cmake; somebody who works on the same project and pulls in your changes would do: runs make -> get undefined references -> hopefully remember to rerun cmake, or files bug with you -> runs cmake -> runs make successfully, whereas if you add file by hand he does: runs make successfully -> spends time with family. Sum that up, don't be lazy, and spare yourself and others a head ache in the future. – sgvd Jan 18 '14 at 13:18
  • yea maybe the real problem is C++, but that is off topic now. – Arne Jan 18 '14 at 22:00
  • @sgvd, Thanks for the write-up! But Coverage and Valgrind cmake instructions are missing with respect to GTest. – uss Jul 17 '17 at 14:52
  • Why is there an imbalance in the block with "configure a header"? It seems like a "`)`" is missing. – Peter Mortensen Nov 12 '22 at 00:49
5

Note: This answer is long because the innocent-looking question and the multiple topics it covers are deceptively broad, and the question's intended audience is for beginners (people new to CMake). Trust me- this answer could have been a lot longer than it is.

What CMake is

Somehow I am totally confused by how CMake works. Every time I think that I am getting closer to understanding how CMake is meant to be written, it vanishes in the next example I read.

CMake is a buildsystem generator. You write a configuration to describe a buildsystem (a project and its build targets and how they should be built) (and optionally, tested, installed, and packaged). You give the CMake program that configuration and tell it what kind of buildsystem to generate, and it generates it (providing that that buildsystem is supported). Such supported buildsystems include (but are not limited to): Ninja, Unix Makefiles, Visual Studio solutions, and XCode.

The buildsystem is the thing that understands (because you instruct it on) how your project needs to be built- what source files it has, and how those source files should get compiled into object files, and how those object files should get linked together into executables or dynamic/shared or static libraries.

The advantages of using CMake should not be understated. If you want to support multiple buildsystems (which is especially common for cross-platform library authors who want to allow their users to make their own buildsystem choices and save those users the work of writing those buildsystem configurations), it is a lot less work to write one CMake configuration than N configurations for N different buildsystems in their own languages and ways of doing things.

Actually, CMake supports other programming languages and their buildsystems than just C and C++, but since you're just asking about C++, I'll leave that out.

Tricks to avoid CMake when using CMake (don't try at home, kids)

All I want to know is, how should I structure my project, so that my CMake requires the least amount of maintainance in the future. For example, I don't want to update my CMakeList.txt when I am adding a new folder in my src tree, that works exactly like all other src folders.

Contrary to what you think, a small degree of having to modify CMakeLists.txt files whenever you add new source files is a very small cost in return for all the benefits of what CMake can provide, and trying to circumvent that cost has its own costs that become problems at scale. That's why those circumvention techniques (namely, file(GLOB)) that people often use are discouraged for use by the maintainers of CMake, and by various long-time users of CMake on Stack Overflow, as seen here and here.

When you have a small project with a few files, it's not very cluttery to list out those few source files in your CMakeLists.txt files, and when you have big projects with lots of source files, you're still better off listing out the source files explicitly for the reasons previously listed in the linked resources. In short, it's for your own good and sanity. Don't try to fight it.

A basic CMake configuration

This is how I imagine my project's structure, but please this is only an example. If the recommended way differs, please tell me, and tell me how to do it.

myProject
    src/
        module1/
            module1.h
            module1.cpp
        module2/
            [...]
        main.cpp
    test/
        test1.cpp
    resources/
        file.png
    bin
        [execute cmake ..]

If you're looking for a convention to use for project filesystem layout, one well-specified layout spec is The Pitchfork Layout Convention (PFL), which was written based on conventions that emerged in the C++ community over time.

Here's what this project layout might look like following the PFL spec with split headers and split tests:

myProject/
  CMakeLists.txt
  libs/
    module1/
      CMakeLists.txt
      include/myProject_module1/
        module1.h
      src/myProject_module1/
        module1.cpp
      tests/
        CMakeLists.txt
        test1.cpp
      data/
        file.png
    module2/
      CMakeLists.txt
      src/module2
        main.cpp
      [...]
  build/

Note: it doesn't have to be include/myProject_module1/- it can just be include/, but adding the myProject_module1/ makes the #includes for each module "namespaced" so that two modules (even if one is from a separate project) can have header files of the same name, and that those headers can all be included in one source file without clashing or ambiguities, like so:

#include <myProject_module1/foo.h>
#include <myProject_module2/foo.h>
#include <yourProject_module1/foo.h>
// Look, ma! No clashing or ambiguities!

Since you allowed so in your question, for the rest of the config code examples, I will use the above PFL layout.

myProject/CMakeLists.txt:

cmake_minimum_required(VERSION 3.25)
# ^choose a CMake version to support (its own can of worms)
# see https://alexreinking.com/blog/how-to-use-cmake-without-the-agonizing-pain-part-1.html
project(example_project
  VERSION 0.1.0 # https://semver.org/spec/v0.1.0.html
  DESCRIPTION "a simple CMake example project"
  # HOMEPAGE_URL ""
  LANGUAGES CXX
)
if(EXAMPLE_PROJECT_BUILD_TESTING)
  enable_testing()
  # or alternatively, `include(CTest)`, if you want to use CDash
  # https://cmake.org/cmake/help/book/mastering-cmake/chapter/CDash.html
endif()
add_subdirectory(libs/module1)
add_subdirectory(libs/module2)
# ^I generally order these from lower to higher abstraction levels.
# Ex. if module1 uses module2, then add_subdirectory it _after_ module2.
# That allows doing target_link_libraries inside moduleN/CmakeLists.txt
# instead of here (although that's equally fine. a matter of preference).

The cmake_minimum_required() command is where you declare what version of CMake is required to parse and run your CMake configuration. The project() command is where you declare the basic project information, the enable_testing() command enables testing for the current directory and below, and each add_subdirectory command changes the "current directory", creates a new subdirectory "scope", and parses the CMakeLists.txt file found at that path.

Here are the docs for cmake_minimum_required(), project(), enable_testing() and add_subdirectory().

myProject/libs/module1/CMakeLists.txt (and similar for module2):

# if module1 is a library, use add_library() instead
add_executable(module1
  src/module1.cpp
)
target_compile_features(okiidoku PUBLIC cxx_std_20) # or whatever language standard you are using
target_include_directories(module1 PUBLIC include)
if(EXAMPLE_PROJECT_BUILD_TESTING)
  add_subdirectory(tests)
endif()

The add_executable() command is how you declare a new executable target to be built. The add_library is similar, but is how you declare a library target to be built, where the library can be linked to via the target_link_libraries() command.

The target_compile_features() command is how you tell CMake what flag to pass to the compiler to pick a C++ language standard to use, and the target_include_directories() command is how you tell CMake what include directories to specify when compiling implementation files. PUBLIC means that the target itself will need that include directory for its #includes, and that other dependent targets that link to that target will as well. If dependent targets don't need it, use PRIVATE. If only dependent targets need it, use INTERFACE. PUBLIC, PRIVATE, and INTERFACE are relevant for library targets, but I'm not aware of them having any use for executable targets (since I'm pretty sure nothing ever depends on executables linkage-wise), so either PUBLIC or PRIVATE should work when specifying include directories for executable targets.

Here are the docs for add_library(), add_executable(), target_compile_features() target_include_directories(), and target_link_libraries().

If you want to learn more about any CMake command (listed in cmake --help-command-list), just do cmake --help <command>, or google cmake command <command>.

In terms of the tests folder, you can use CMake's testing support without using any C++ testing libraries of frameworks, or use it with a testing library or framework that supports CMake. To read more about CMake and testing in general, read the chapter in the Mastering CMake book. It's too much material to cover in the form of a Stack Overflow answer.

Installation is also its own can of worms (more on that later), so since the question didn't ask for it, I think it's better to leave out of the answer post to avoid an super long post and scope creep. Again, see the dedicated chapter in the Mastering CMake book. One thing to especially watch out for with installation is making sure you make the install package relocatable.

In terms of a very very simple project, that's all you need, although of course, there can be much more configuration to do based on your project's specific needs, but you can burn yourself on those bridges when you get there.

Compile options / flags is also its own can of worms and better covered in separate Q&A posts.

If you want to start using dependencies, I suggest reading the official "Using Dependencies" guide.

If you really have a lot of source files, and your myProject/src/module1/CMakeLists.txt file starts to get unwieldy because of all the lines of add_executable/add_library, then you can factor that out into a separate file using target_sources() and either another CmakeLists.txt file in a subdirectory included via add_subdirectory, or a sources.cmake file in the same subdirectory included via include().

To generate the buildsystem as shown in your directory tree diagram, change your current directory to .../myProject and then run cmake -S . -B build <...>, where <...> is any other configuration arguments you want to use.

As always, the CMake reference docs can be quite helpful, but overwhelming to look at the first couple of times, since they're not meant for beginners to learn from. If you want to learn more about how to use CMake, try out the official CMake tutorial, and reading relevant chapters in the Mastering CMake book. If you really want to dive deep, check out "Professional CMake"- written by Craig Scott (one of the CMake maintainers). It costs money, but having read the sample chapter, the table of contents, and other blog posts and proposals on the CMake GitLab by Craig, I have faith in its value, and new editions to the book don't cost extra.

Tricks for Resource Files

By the way, it is important that my program knows where the resources are. I would like to know the recommended way of managing resources. I do not want to access my resources with "../resources/file.png"

This response is written assuming you chose CMake to use it for all it's worth- cross-platform, flexible-toolchain builds, which not everybody uses CMake for (which is fine).

This is its own can of worms. One tricky part is the filesystem placement compared between the build folder and the install folder. The build folder is where the binaries and other ingredients like object files get built, and the install folder is anywhere you install those built binaries to. They can have very different filesystem structures, where the build folder layout is up to CMake, and it does sensible things by default, but the layout of things in the install folder is largely up to how you want it to be configured. That can be different than what CMake does in the build folder, and you'll probably want to be able to run and test your binaries from both the build folder and the install folder. So you need to find a way to support your binaries finding your resource files in both where they are at "development time" (when you're running from the build folder), and after installation (when you or your users are running the installed binaries).

There are also various different conventions on different platforms for where to place resource files for installation. There's a convention defined by the GNU Coding Standards, which CMake has a bit of integration with / support for, but of course, Microsoft Windows has another thing with \Program Files\ and \Users\...\AppData\, and MacOS has another thing with app bundles and /library/Application Support/. Things might not be as simple as you thought they were. I don't know a lot about this (and I could be wrong), but it seems to me that this is big enough of a topic to have its own question, or several of its own questions here on Stack Overflow.

For other related CMake bits, see:

For examples of shortcomings with simple/naive approaches, the answer by sgvd works for the build directory, but will not work wherever the project is installed to, and even if it's made to somehow work on the builder's machine, the fact that it uses an absolute path makes it unlikely to work when distributed to other machines or platforms with different conventions.

If you're using CMake but only want to support a specific platform, then consider yourself lucky and find or write a question here on Stack Overflow about how to do that.

See also this related question (which at the time of this writing still has no answers): How to specify asset paths that work across builds.

Send-off

Welcome to the world of CMake! Just wait 'till you get to generator expressions! Then you'll really start having fun!

^said in jest, but no joking- generator expressions can be very useful, and I'd choose them any day over suffering in the Visual Studio configurations UI or manually editing Visual Studio solution files (just to give an example for one buildsystem).

starball
  • 20,030
  • 7
  • 43
  • 238
  • Wow, this answer is a surprise. While I currently don't use CMake in my life anymore (yay), you pretty much help me to put into words why I hate it so much. Everything is see as "basic" and should just work out of the box is "its own can of worms", things I want are "discouraged for use" and things I don't care about are explained in overwhelming details in the tutorials and hard to ignore. It feels like a constant brushing the wrong way. Still reading your answer was a joy. Thank you. – Arne Jan 13 '23 at 18:00
  • @Arne In my understanding, the complexity of resource files and builds and installs isn't CMake's "fault". It's just that the world is complex when it comes to that, so don't "put blame" on CMake for that part. Similar can be said for project layout. That's not really CMake's fault either. It's just that _C++_ was _designed_ to be very flexible with respect to that instead of being perscriptive. Similar for test frameworks, installation on various platforms, etc. – starball Jan 13 '23 at 19:19
  • It seems to me like a fairly large portion of what CMake gets hated on for is scape-goating it for the complexity outside of itself that it bears the burden of being an interface to. Don't get me wrong- there _are_ ways that CMake can be improved. Just go browse [popular issue tickets on the KitWare GitLab repo](https://gitlab.kitware.com/cmake/cmake/-/issues/?sort=popularity&state=opened). For example, [this one about exporting](https://gitlab.kitware.com/cmake/cmake/-/issues/18634). – starball Jan 13 '23 at 19:24
  • the point is, when starting a project, all this flexibility is just flexibility to shoot yourself in the food. You don't want a tutorial that explains you 10 variants that you could technically use to write test cases in different structures all with ups and downs. You need a folder where to put the tests, and structure on how to write them. For the most part I am much happier writing `go` programs that I was writing C++. It just works. – Arne Jan 13 '23 at 19:27