CMake ADD_CUSTOM_COMMAND with unknown output files

Question

I have a tool to generate some cpp and header files and want to add it with ADD_CUSTOM_COMMAND to automatically execute it during the build and add the files to the project. The problem is that the names of (most of) the output files are not known beforehand. How do I add those files?

Answer 1

Introduction

I assume that your goal is not the usage of ADD_CUSTOM_COMMAND, but rather that:

• cmake recognizes that your generated files depend on the generation script. This means: when your generation script has changed, then the files are regenerated automatically with different file names when you execute cmake --build the next time.
• cmake recognizes that your executable/library depends on the generated files, whose file names change as soon as these are regenerated. This means: your executable/library is rebuilt and uses the regenerated files as consequence of the same call of cmake --build

The main problem is that filenames must be known at configuration time, so that the dependencies are known at build time. This means that you need to move the generation of the files to configuration time.

You either need to get the file names as output of your generation script or you use file globbing. Usually I do not recommend the usage of file globbing, because it has some disadvantages regarding proper dependencies. In your case globbing adds real value, but the disadvantage must be compensated.

Below I compiled a complete example project which shows how it could work. I do not say that this is the standard way to do, but I think that it meets the requirements well.

Examplary Setup

I have created a minimum setup which makes all this working:

myProject
|- CMakeLists.txt
|- main.cpp
|- myClass.h
|- generate.sh

myClass.h

#ifndef MY_CLASS_H
#define MY_CLASS_H
int do_something_a();
int do_something_b();
#endif // MY_CLASS_H

main.cpp

#include "myClass.h"
#include <iostream>

int main(const int argc, const char **argv)
{
  int a = do_something_a();
  int b = do_something_b();
    
  std::cout << "A:" << a << ", B:" << b << std::endl;

  return 0;
}

generate.sh Just creates 2 cpp files which implement the do_something_a() and do_something_b(). The generated file names contain a random number, which is also returned by the do_something_*() function. The files are put into subdirectory generated.

#!/usr/bin/env bash

# just creating a new subfolder; remove its contents if already existing
dir1="generated"
if [ ! -d ${dir1} ]; then mkdir ${dir1}; fi
rm -rf ${dir1}/*

# just create a cpp file with random name and implement do_something_a()    
var1=`date +%N`
file1=${dir1}/myClass.${var1}.cpp
echo '#include "myClass.h"' > ${file1}
echo 'int do_something_a() { return '${var1}'; }' >> ${file1}

# just create a cpp file with random name and implement do_something_b()
var2=`date +%N`
file2=${dir1}/myClass.${var2}.cpp
echo '#include "myClass.h"' > ${file2}
echo 'int do_something_b() { return '${var2}'; }' >> ${file2}

CMakeLists.txt

cmake_minimum_required (VERSION 3.10.2)

project(useGenerateSourcesAtBuildStepProj
  VERSION 0.0.0.1
  DESCRIPTION "Minimum Example: Use Generated Files as dependency"
  LANGUAGES CXX
)

# just needed to force re-configuration, if generate.sh has changed.
# that's the tweak to overcome glob disadvantage
configure_file(
  ${CMAKE_CURRENT_SOURCE_DIR}/generate.sh
  ${CMAKE_CURRENT_BINARY_DIR}/generate.sh
)

# regeneration command executed at configuration time
# note: cannot call ${CMAKE_CURRENT_BINARY_DIR}/generate.sh, because not yet existing
execute_process(
  COMMAND bash ${CMAKE_CURRENT_SOURCE_DIR}/generate.sh
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)

# globbing all generated files in sub directory `generated` at configuration time after execute_process
FILE(GLOB GENERATED_SRC_FILES ${CMAKE_CURRENT_BINARY_DIR}/generated/*)

# use generated files to build your executable
add_executable(useGenerateSourcesAtBuildStep main.cpp ${GENERATED_SRC_FILES})
target_include_directories(useGenerateSourcesAtBuildStep PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

Build

Configure:

../bin/Release$ cmake -G"Unix Makefiles" ../..

Output:

-- The CXX compiler identification is GNU 7.5.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done    
-- Configuring done
-- Generating done
-- Build files have been written to: <dir>/bin/Release

Build:

../bin/Release$ make

Output:

Scanning dependencies of target useGenerateSourcesAtBuildStep
[ 25%] Building CXX object CMakeFiles/useGenerateSourcesAtBuildStep.dir/main.cpp.o
[ 50%] Building CXX object CMakeFiles/useGenerateSourcesAtBuildStep.dir/generated/myClass.488106246.cpp.o
[ 75%] Building CXX object CMakeFiles/useGenerateSourcesAtBuildStep.dir/generated/myClass.490335510.cpp.o
[100%] Linking CXX executable useGenerateSourcesAtBuildStep
[100%] Built target useGenerateSourcesAtBuildStep

Generated Files:

|- bin
    |- Release
        |- src
            |- generate.sh (due to configure_file)
            |- generated (directory containing all generated files)
            |   |- myClass.488106246.cpp
            |   |- myClass.490335510.cpp
            |- useGenerateSourcesAtBuildStep (your exectuable)

Change generate.sh and rebuild with make:

../bin/Release$ make

Output:

You can see that the configuration step is done again and that the generated files change their name, but are rebuild before linking it all together into your executable:

-- Configuring done
-- Generating done
-- Build files have been written to: /home/tangoal/Documents/data/300_Projects/_software/CPP/lib/test/bin/Release
Scanning dependencies of target useGenerateSourcesAtBuildStep
[ 25%] Building CXX object CMakeFiles/useGenerateSourcesAtBuildStep.dir/generated/myClass.565834060.cpp.o
[ 50%] Building CXX object CMakeFiles/useGenerateSourcesAtBuildStep.dir/generated/myClass.567160574.cpp.o

[ 75%] Linking CXX executable useGenerateSourcesAtBuildStep
[100%] Built target useGenerateSourcesAtBuildStep

Execute the program:

../Release/src$ ./useGenerateSourcesAtBuildStep

Output:

A:565834060, B:567160574