24

I have several projects (all building with CMake from the same source tree structure) all using their own mix out of dozens of supporting libraries.

So I came about the question how to set up this correctly in CMake. So far I have only found CMake how to correctly create dependencies between targets, but I'm still struggling between setting up everything with global dependencies (the project level does know it all) or with local dependencies (each sub-level target only handles its own dependencies).

Here is a reduced example of my directory structure and what I currently came up with using CMake and local dependencies (the example shows only one executable project, App1, but there are actually more, App2, App3, etc.):

Lib
+-- LibA
    +-- Inc
        +-- a.h
    +-- Src
        +-- a.cc
    +-- CMakeLists.txt
+-- LibB
    +-- Inc
        +-- b.h
    +-- Src
        +-- b.cc
    +-- CMakeLists.txt
+-- LibC
    +-- Inc
        +-- c.h
    +-- Src
        +-- c.cc
    +-- CMakeLists.txt
App1
+-- Src
    +-- main.cc
+-- CMakeLists.txt

Lib/LibA/CMakeLists.txt

include_directories(Inc ../LibC/Inc)
add_subdirectory(../LibC LibC)
add_library(LibA Src/a.cc Inc/a.h)
target_link_libraries(LibA LibC)

Lib/LibB/CMakeLists.txt

include_directories(Inc)
add_library(LibB Src/b.cc Inc/b.h)

Lib/LibC/CMakeLists.txt

include_directories(Inc ../LibB/Inc)
add_subdirectory(../LibB LibB)
add_library(LibC Src/c.cc Inc/c.h)
target_link_libraries(LibC LibB)

App1/CMakeLists.txt (for the ease of reproducing it I generate the source/header files here)

cmake_minimum_required(VERSION 2.8)

project(App1 CXX)

file(WRITE "Src/main.cc" "#include \"a.h\"\n#include \"b.h\"\nint main()\n{\na();\nb();\nreturn 0;\n}")
file(WRITE "../Lib/LibA/Inc/a.h" "void a();")
file(WRITE "../Lib/LibA/Src/a.cc" "#include \"c.h\"\nvoid a()\n{\nc();\n}")
file(WRITE "../Lib/LibB/Inc/b.h" "void b();")
file(WRITE "../Lib/LibB/Src/b.cc" "void b() {}")
file(WRITE "../Lib/LibC/Inc/c.h" "void c();")
file(WRITE "../Lib/LibC/Src/c.cc" "#include \"b.h\"\nvoid c()\n{\nb();\n}")

include_directories(
    ../Lib/LibA/Inc
    ../Lib/LibB/Inc
)

add_subdirectory(../Lib/LibA LibA)
add_subdirectory(../Lib/LibB LibB)

add_executable(App1 Src/main.cc)

target_link_libraries(App1 LibA LibB)

The library dependencies in the above example do look like this:

App1 -> LibA -> LibC -> LibB
App1 -> LibB

At the moment I prefer the local dependencies variant, because it's easier to use. I just give the dependencies at the source level with include_directories(), at the link level with target_link_libraries() and at the CMake level with add_subdirectory().

With this you don't need to know the dependencies between the supporting libraries and - with the CMake level "includes" - you will only end-up with the targets you really use. Sure enough you could just make all include directories and targets be known globally and let the compiler/linker sort out the rest. But this seems like a kind of bloating to me.

I also tried to have a Lib/CMakeLists.txt to handle all the dependencies in the Lib directory tree, but I ended up having a lot of if ("${PROJECT_NAME}" STREQUAL ...) checks and the problem that I can't create intermediate libraries grouping targets without giving at least one source file.

So the above example is "so far so good", but it throws the following error because you should/can not add a CMakeLists.txt twice:

CMake Error at Lib/LibB/CMakeLists.txt:2 (add_library):
  add_library cannot create target "LibB" because another target with the
  same name already exists.  The existing target is a static library created
  in source directory "Lib/LibB".
  See documentation for policy CMP0002 for more details.

At the moment I see two solutions for this, but I think I got this way too complicated.

1. Overwriting add_subdirectory() to prevent duplicates

function(add_subdirectory _dir)
    get_filename_component(_fullpath ${_dir} REALPATH)
    if (EXISTS ${_fullpath} AND EXISTS ${_fullpath}/CMakeLists.txt)
        get_property(_included_dirs GLOBAL PROPERTY GlobalAddSubdirectoryOnceIncluded)
        list(FIND _included_dirs "${_fullpath}" _used_index)
        if (${_used_index} EQUAL -1)
            set_property(GLOBAL APPEND PROPERTY GlobalAddSubdirectoryOnceIncluded "${_fullpath}")
            _add_subdirectory(${_dir} ${ARGN})
        endif()
    else()
        message(WARNING "add_subdirectory: Can't find ${_fullpath}/CMakeLists.txt")
    endif()
endfunction(add_subdirectory _dir)

2. Adding an "include guard" to all sub-level CMakeLists.txts, like:

if (NOT TARGET LibA)
    ...
endif()

I've been testing the concepts suggested by tamas.kenez and m.s. with some promising results. The summaries can be found in my following answers:

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Florian
  • 39,996
  • 9
  • 133
  • 149
  • I don't see the reason for `add_subdirectory(../Lib/LibA LibA)` within **App1/CMakeLists.txt**. Why do you need this? Usually you have a root *CMakeLists.txt* which adds the subdirectories. – m.s. Jul 20 '15 at 09:33
  • @m.s. in my real world project there are more applications then just my `App1` (I added some details about this to the question). So far I didn't add a root level `CMakeLists.txt` to prevent having a dependency between the different project teams working an `App1`, `App2`, etc (and I couldn't think of one that is on one site generic enough for all projects and on the other site specific enough that at the end it generates only one executable target). At the moment they just go to their subdirectory, create a binary subdirectory, generate their make environment with `cmake ..` and call `make`. – Florian Jul 20 '15 at 10:25
  • How about adding `LibA`, `LibB`, etc. through [`ExternalProject_Add`](http://www.cmake.org/cmake/help/v3.2/module/ExternalProject.html) when working on `App1`? – m.s. Jul 20 '15 at 10:29
  • @m.s. In my `App1` example if I add both `LibA` and `LibB` via external projects, wouldn't I end up with duplicate symbols for everything that is in `LibB`? – Florian Jul 20 '15 at 10:34
  • have a look at the article series here: https://coderwall.com/p/qk2eog/use-cmake-enabled-libraries-in-your-cmake-project-ii – m.s. Jul 20 '15 at 10:36

1 Answers1

19

Adding the same subdirectory multiple times is out of question, it's not how CMake is intended to work. There are two main alternatives to do it in a clean way:

  1. Build your libraries in the same project as your app. Prefer this option for libraries you're actively working on (while you're working on the app) so they are likely to be frequently edited and rebuilt. They will also show up in the same IDE project.

  2. Build your libraries in an external project (and I don't mean ExternalProject). Prefer this option for libraries that are just used by your app but you're not working on them. This is the case for most third-party libraries. They will not clutter your IDE workspace, either.

Method #1

  • your app's CMakeLists.txt adds the subdirectories of the libraries (and your libs' CMakeLists.txt's don't)
  • your app's CMakeLists.txt is responsible to add all immediate and transitive dependencies and to add them in the proper order
  • it assumes that adding the subdirectory for libx will create some target (say libx) that can be readily used with target_link_libraries

As a sidenote: for the libraries it's a good practice to create a full-featured library target, that is, one that contains all the information needed to use the library:

add_library(LibB Src/b.cc Inc/b.h)
target_include_directories(LibB PUBLIC
    $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/Inc>)

So the location of include directories of the library can remain an internal affair of the lib. You will only have to do this;

target_link_libraries(LibC LibB)

then the include dirs of LibB will also be added to the compilation of LibC. Use the PRIVATE modifier if LibB is not used by the public headers of LibC:

target_link_libraries(LibC PRIVATE LibB)

Method #2

Build and install your libraries in seperate CMake projects. Your libraries will install a so-called config-module which describes the locations of the headers and library files and also compile flags. Your app's CMakeList.txt assumes the libraries has already been built and installed and the config-modules can be found by the find_package command. This is a whole another story so I won't go into details here.

A few notes:

  • You can mix #1 and #2 as in most cases you will have both unchanging, third-party libs and your own libraries under development.
  • A compromise between #1 and #2 is using the ExternalProject module, preferred by many. It's like including the external projects of your libraries (built in their own build tree) into your app's project. In way it combines the disadvantages of both approaches: you can't use your libraries as targets (because they're in a different project) and you can't call find_package (because the libs are not installed the time your app's CMakeLists is configuring).
  • A variant of #2 is to build the library in an external project but instead of installing the artifacts use them from their source/build locations. For more about this see the export() command.
tamas.kenez
  • 7,301
  • 4
  • 24
  • 34
  • *Method 1*: I like your use of the `target_include_directories()` command (CMake >= 2.8.12), it's definitely makes things easier. But I'm searching for a solution where the user of the libraries doesn't need to know the internal dependencies. *Method2*: I will take a look into using binary delivery and the `find_package()` command. Can you recommend any useful links? – Florian Jul 21 '15 at 07:10
  • Regarding the `ExternalProject` module: I think the command `ExternalProject_Add()` is a possibility if we are talking about larger monolithic libraries. But for my 50+ smaller libraries I don't think this is an option and to a certain degree the argument that it's not meant to be used that way holds true here too. The overhead is just to big: e.g. I'm concerned that it - if I would use the `SOURCE_DIR` option - would start compiler detection for every library and I have to pass along all build environment specific options incl. my toolchain file. – Florian Jul 21 '15 at 07:15
  • 2
    @Florian, for method2 I created a tutorial question: http://stackoverflow.com/questions/31537602/how-to-use-cmake-to-find-and-link-to-a-library-using-install-export-and-find-pac . It covers only a single library without dependencies. I may add a new tutorial in the next days which introduces dependencies. Or you can also put up a new question about that one. – tamas.kenez Jul 21 '15 at 11:28
  • @Florian, ExternalProject: You're right about the overhead. For the other reasons I mentioned I don't use it either. – tamas.kenez Jul 21 '15 at 11:30
  • 1
    @Florian, managing 50+ small libs: to effectively do this you will need some additional infrastructure, like a set of custom or generic shell scripts. Many people created entire frameworks around this (biicode, cmakepp, CPM, fips, hunter). I wrote a CMake-based package/repo manager for myself. – tamas.kenez Jul 21 '15 at 11:35
  • After testing - over the last couple of months - your suggested concepts, I admit I came to the same conclusions. More generally speaking your CMake environment should fit to your processes/workflow and should take project sizes and team structure into account. See my answers [here](http://stackoverflow.com/questions/33534115/preferred-cmake-project-structure) and [here](http://stackoverflow.com/questions/33443164/cmake-share-library-with-multiple-executables). – Florian Nov 14 '15 at 21:39