7

Consider two software projects, proj_a and proj_b, with the latter depending on the former; and with both using CMake.

When reading about modern CMake, one gets the message that the "appropriate" way to express dependencies is via target dependencies; and one should arrange it so that dependent projects are represented as (imported) targets you can depend on. More specifically, in our example, proj_b will idiomatically have:

find_package(proj_a)

# etc etc.

target_link_library(bar proj_a::foo)

and proj_a will need to have been installed, utilizing the CMake installation-and-export-related commands, someplace where proj_b's CMake invocation will search for proj_a-config.cmake.

I like this approach and encourage others to adapt to it. It offers flexibility in the choice of your own version of proj_a vs the system version; and also allows for non-CMake proj_a's via a Findproj_a.cmake script (which again, can be system-level or part of proj_b).

So far so good, right? However, there are people who want to "take matters into their own hands" in terms of dependencies - and CMake officially condones this, with commands such as ExternalProject and more recently, FetchContent: This allows proj_b's configuration stage to actually download a (built, or in our case source-form) version of proj_a.

The puzzling part to me is that, after proj_a is downloaded, say to an external/proj_a directory, CMake's default behavior will be to

add_subdirectory(external/proj_a)

that is, to use proj_a as a subproject of proj_b and build them together. This, while the idiomatic use above allows the maintainer of proj_a to "do their own thing" in my CMakeFile, and only keep things neat and tidy for others via what I export/install.

My questions:

  1. Why does it make sense to add_subdirectory(), rather than to build, install, and perform the equivalent of find_package() to meet the dependency? Or rather, why should the former, rather than the latter, be the default?
  2. Should I really have to write my project-level CMakeLists.txt to be compatible with being add_subdirectory()'ed?

Note: Just to give some concrete examples of how this use constrains proj_a:

  • Must use unique option names which can't possibly clash with super-project names. So no more WITH_TESTS, BUILD_STATIC_LIB - it has to be: WITH_PROJ_A_TESTS and BUILD_PROJ_A_STATIC_LIB.
  • You have to account for the parent project having searched for other dependencies already, and perhaps differently than how you would like to search for them.
einpoklum
  • 118,144
  • 57
  • 340
  • 684
  • "Why does it make sense to `add_subdirectory()`, rather than to build, install, and perform the equivalent of `find_package()`" - It depends on your specific case. Usually you choose the former, but the latter is also usable. However, I find *installing* one project during configuration of another project to be something strange. E.g. configuration usually doesn't require root privileges, but system installation requires them. – Tsyvarev Nov 12 '21 at 09:42
  • "Should I really have to write my project-level CMakeLists.txt to be compatible with being add_subdirectory()'ed?" - It is up to you. I see more and more people who **expect** other projects to be usable via `add_subdirectory`. (Aren't you among these people?) – Tsyvarev Nov 12 '21 at 09:44
  • @Tsyvarev: "depends on your use case" sounds reasonable, but CMake defaults to just `add_subdirectory()`; it doesn't let the proj_a author say "Please use my installation result version, not my sources". As for installing during configuration - I agree this is a bit strange, but, in principle, you can't know how proj_a can even be _used_ until _after_ it has been built and installed. Plus, downloading stuff over the internet during project configuration is already a bit stange IMHO. – einpoklum Nov 12 '21 at 09:49
  • "but CMake defaults to just `add_subdirectory()`" - Not quite true. The only function from FetchContent module which does that is `FetchContent_MakeAvailable`. But by using `FetchContent_Populate` one can implement any other population logic. Note also, that documentation for FetchContent clearly states that fetched project is not required to have `CMakeLists.txt` at all. – Tsyvarev Nov 12 '21 at 09:57
  • @Tsyvarev: I didn't say "not allow", I said "defaults"... one can usually implement other things yourself. The question is, why this default? – einpoklum Nov 12 '21 at 10:05
  • Your question seems to be about general "Would I prefer `find_package` or `add_subdirectory` for include another project into mine?". I know no generic algorithm for choose between these approaches. Including large and complex projects (e.g. QT or Boost) via `add_subdirectory` would greatly pollute CMake "environment" of your project, and makes configuration of subproject to be harder. On the other side, including googletest for testing your project via `add_subdirectory` looks nice and simple. – Tsyvarev Nov 12 '21 at 10:31
  • In any case, the approach with `add_subdirectory` becomes more and more popular, and module `FetchContent` provides additional facilities for that approach: Downloading or cloning the other project, ensuring that download/clone happens only once, etc. While there facilities can be used for any other purposes, the **main usage** of FetchContent is exactly for `add_subdirectory`. So it is not FetchContent which encourages using of `add_subdirectory`. It is popular `add_subdirectory` which becomes more convenient when used with FetchContent. – Tsyvarev Nov 12 '21 at 10:43
  • Just a sidenote: "Must use unique option names" -> No you don't and you shouln't in my opinion. Variables have local scope, therefore a set(BUILD_TESTING ON) won't affect the parent scope. If proj_b wants to turn BUILD_TESTING OFF just for that project then (or in general) it should call FetchContent_MakeAvailable in a subdirectory or in function. This won't then affect proj_b (for the same reasons). By using the standard names people don't have to look up what you specific variable name is. – Leon0402 Nov 13 '21 at 15:18
  • Some more quick tips: - Turn `BUILD_TESTING` and similar things OFF by default if included as a subdirectory. You can check whether the current CMake File is the top level file - Don't set things like `BUILD_STATIC_LIB`. There is a standard variable called `BUILD_SHARED_LIBS`. If you just specify `add_library(MyLibrary)` it will be static or shared dependening on that variable. So just have one target for both! Let the user decide by passing `BUILD_SHARED_LIBS` themself (which then works for all included libraries that use that pattern!) – Leon0402 Nov 13 '21 at 15:18
  • @Tsyvarev: "the main usage of FetchContent is exactly for add_subdirectory" <- IMHO, that is a chicken-and-egg situation. FetchContent is geared towards add_subdirectory, and nobody has been encouraged to try it differently, so... – einpoklum Nov 13 '21 at 18:22
  • @Leon0402: Doesn't subdirectory-scoping apply only to non-cache variables? – einpoklum Nov 13 '21 at 18:24
  • @einpoklum It's a little bit more compley and I hope I get it right. A `set( )` will in general always set a normal variable and not modify any cache variable. A normal variable will shadow the cache variable. Basically that's the trick here. But (!): A set(Variable CACHE ...) will remove on the first run any local variable. So in the first run of cmake the cache variable will be taken. option() on the other hand was changed with CMake 3.13 and behave like you expect. So setting a normal variable works perfectly, even on the first run. – Leon0402 Nov 14 '21 at 15:13
  • So yeah it basically only works for boolean variables set with option(), which most interesting variables you want to override are. If you really have some non boolean / option cache variable you can always force set and unset afterwards (or save the old value before calling add_subdirectory) – Leon0402 Nov 14 '21 at 15:19
  • @Leon0402: Well, the user-settable options _are_ cache variables. It doesn't matter whether I can override them or not, that's not the problem I was bringing up. – einpoklum Nov 14 '21 at 19:52
  • @einpoklum I was arguing against your point "Must use unique option names which can't possibly clash with super-project names". Most people think thism because they believe they cannot override it (without affecting the own variables), therefore I explained how to do it. If this isn't you problem, why do you think then that these cache variables have to be unique? (If you argument is: Your cache variables affect the dependencies. That is usually wanted. If you don't want this set a specific value as explained before) – Leon0402 Nov 15 '21 at 08:12
  • Or to be more general: Libraries written in modern CMake are almost always already compatible with FetchContent. There are two exceptions: - One should disable certain options by default if it's not the top level project - One should provide alias targets Both of them are very easy to implement. Most other problems (I saw up to know) came from using things you shouldn't be doing anyway :-) I saw in the bug that you have a library where FetchContent support came up. If you want to, leave a link and I will have a look what the specific problem is there – Leon0402 Nov 15 '21 at 08:19

2 Answers2

3

Following the discussion in comments, I decided to post a bug report about this:

#22904: Support FetchContent_MakeAvailable performing build+install+find_package rather than add_subdirectory

So maybe this will change and the question becomes moot.

einpoklum
  • 118,144
  • 57
  • 340
  • 684
  • 2
    See my response in the linked CMake issue, which hopefully sheds some light on the bigger picture around this area (I'm the author of FetchContent). – Craig Scott Nov 14 '21 at 04:10
1

Why does it make sense to add_subdirectory(), rather than to build, install, and perform the equivalent of find_package() to meet the dependency? Or rather, why should the former, rather than the latter, be the default?

FetchContent doesn't just have to be for project() dependencies. It can be used for fetching utility scripts too. I'm guessing it was designed with that kind of consideration in mind. If your utility script is just one file, you can just file(DOWNLOAD) and add_subdirectory() directly, but the utilities could be multiple files, such as is the case with aminaya/project_options. FetchContent() uses a lot of the same machinery as ExternalProject, so it can do a lot of the useful things that ExternalProject does. For example, you can use FetchContent to fetch aminaya/project_options as a remote git repo, or as its archive artifacts- ex. v0.20.0.zip

Should I really have to write my project-level CMakeLists.txt to be compatible with being add_subdirectory()'ed?

It's your choice! The reasoning here can be highly objective, or subjective. It's up to you. Some people just like to put in a lot of effort to support whatever their users might want. Some people have a lot of historical configuration baggage and are still catching up to newer CMake. And as you mentioned at the end of your question post, there are certain adjustments that need to be made to accomodate for cleanly allowing people to add_subdirectory() you as a dependency. One example of a project which chose "no" is glew (see issue #314 for explanation).

Just to give another reference to some related work mentioned in responses to the KitWare/CMake ticket your raised, here's the ticket which tracked work on "FetchContent and find_package() integration".

starball
  • 20,030
  • 7
  • 43
  • 238