0

There are several other similar questions 1 2 3 4 5, but none answer this question / usecase. (I also heard about ExternalProject_Add, but what I heard suggested it wouldn't solve this find_package problem).

I have several 3rd party CMake/C++ libraries, which I'm to gather into one repo, and compile together. Ex:

CMakeLists.txt // add_subdirectory(foo)
               // set(foo_DIR ${CMAKE_CURRENT_BINARY_DIR}/foo)
               // add_subdirectory(bar)
| foo/
| | CMakeLists.txt
| | include/, src/, etc.
| bar/
| | CMakeLists.txt // find_package(foo)
| | include/, src/, etc.

This looks simple, but both foo and bar have large CMakeLists.txt's, find-pkg'ing several other libraries, and I have multiple libraries in this style, where I'd like to compile together. If possible, I'd prefer to only change the top-level CMakeLists.txt, and not change the subdirectories/projects.

If I compile this from a blank build/ dir, I get the error build/foo/foo-config.cmake could not find the requested file build/foo/foo-targets.cmake, from bar/CMakeLists.txt:38 find_package (foo). Hoping this was just a build order issue, and specifying the foo_DIR was sufficient, I commented out the #add_subdirectory(bar), successfully built foo, then uncommented out addsubdirectory and successfully built bar. (This requires the set(foo_DIR), to find the output cmake file, which came from this question, but didn't fix the simultaneous compile).

Is there a way to control the subdirectory compilation order, or something, so dependent subprojects' find_package still works unmodified?

JWCS
  • 1,120
  • 1
  • 7
  • 17
  • what happens if you set `foo_DIR` to some non-empty string before the call to `add_subdirectory(bar)`? Does foo define alias targets matching its exported namespaced targets? – starball Jun 14 '23 at 16:42
  • If foo outputs cmake targets to build/foo/, and it's not elsewhere on the system, then find_package won't be able to find foo. The add_subdirectory(bar) has a CMakeLists that find_package(foo)'s , which can pick up the foo_DIR. (As far as I understand). At the least, removing foo_DIR stops foo from being found. – JWCS Jun 14 '23 at 19:43
  • I'm not sure if you understood what I was trying to get at. I'm suggesting to try short-circuiting the find_package work from ever being done by tricking it into thinking it has already finished successfully. – starball Jun 14 '23 at 19:50

3 Answers3

1

Generally speaking, you don't need to find_package something that has been add_subdirectory-ed. add_subdirectory adds something to be part of the same configured and generated buildsystem. find_package is for things that are already built and don't need buildsystem rules generated for. If you want to control the order of two targets being built relative to one another, use add_dependencies.

starball
  • 20,030
  • 7
  • 43
  • 238
  • True, which I would absolutely do if they were my own libraries, but since they're not, I'm trying to avoid modifying them so I can pull upstream changes more simply. If there was a native cmake solution, that'd be the best... – JWCS Jun 14 '23 at 14:38
1

Your problem is basically that you'd need the subdirectory to be fully compiled and installed by the time find_package is called. This is not easily possible since the basic flow of cmake is:

  1. configure
  2. build/install

What you're asking for is:

  1. partially configure
  2. build the previously configured parts
  3. continue configuring where we left off

This is not going to work without seriously ugly workarounds.

The sane solution

Fix it in the subdirectories. They should be checking if a dependency is already available as a cmake target before calling find_package. I know you don't want to make local modifications, but this kind of thing should be pushed upstream anyway, because it's the "correct" behavior.

Mostly insane workarounds

  1. Build and install the subdirectories via script. You could, during cmake configuration, execute custom shell commands to build the subdirectories one by one. At that point it might be reasonable to just use a custom script instead of cmake, since you're circumventing cmake pretty hard anyway.
  2. Build a custom find script that doesn't actually look for the installed package but instead simply checks if the required targets are available and sets (pkgname)_FOUND. Whether this works depends on how the subdirectories actually include their dependencies. If they use targets then good, if they *_LIBRARIES and other such variables then you'll have to do a lot of fiddling to make it work, if at all.
Wutz
  • 2,246
  • 13
  • 15
  • Yup; as much as I wanted to avoid changing the subdirs/subrepos, some modifications were required anyway... I did attempt the scripting around the compiling process, which I may post as an answer, but it was much more time then just commenting out the find_packages... – JWCS Jun 15 '23 at 18:31
  • Did it work by simply commenting them out? If so a workaround without changes might be possible. For example by setting (pkgname)_FOUND to true – Wutz Jun 15 '23 at 20:15
  • Setting pkgname_FOUND true doesn't change the behavior of find_package; it tries to find_package anyway. If you only comment out the find_package lines, it results in errors at target_link_libraries, where the target lib::lib is not found; this is provided by find_package. This is similar to the crazy answer I put below, where you can make empty -config.cmake and -targets.cmake files, but it only works (so far) for header only libraries, or vendoring libraries that don't actually provide any targets. – JWCS Jun 15 '23 at 20:56
0

Insane Solution that may partially work in certain scenarios

I did have the case where a few of my dependencies were just vendoring cmake packages, and I was able work around it trivially. For example, a package had both find_package(eigen3 REQUIRED) and the find_package(eigen3_cmake_module REQUIRED) vendoring package. If eigen3 can be found already, the find_package can be spoofed by feeding it a fake (empty) eigen3_cmake_module-config.cmake file, pointed to in the cmakelists by set(eigen3_cmake_module_DIR <abs_path_to_dir>). Ex:

mkdir -p cmake
touch cmake/eigen3_cmake_module-config.cmake
echo "set(eigen3_cmake_module_DIR \${CMAKE_CURRENT_SOURCE_DIR}/cmake)" >> CMakeLists.txt

For real libraries, they'll complain that there's no *-targets.cmake file, but if that's empty, at target_link_libraries, the target may not be found. This is the kind of foolery @starball was getting at, and @Wutz alluded to as bad solution 2. If dead set on this, one deeper thing I did not explore was the cmake 2.24 find_package new option to force re-route the search:

New in version 3.24: All calls to find_package() (even in Module mode) first look for a config package file in the CMAKE_FIND_PACKAGE_REDIRECTS_DIR directory. The FetchContent module, or even the project itself, may write files to that location to redirect find_package() calls to content already provided by the project. If no config package file is found in that location, the search proceeds with the logic described below.

JWCS
  • 1,120
  • 1
  • 7
  • 17