Custom build step after preprocessing and before compilation with CMake

Question

I'm trying to add a custom build step between preprocessing and compiling using CMake.

After the c++ preprocessor step, I want to call a python script on each preprocessed source files to modify them.

Here a Makefile example:

all : prog

# compilation step
prog: main.i
    gcc main.i -g -Wall -o prog 

# custom step
main.i: main.tmp
    ./my_script.py main.tmp > main.i

# only preprocessor step
main.tmp: main.c main.h
    gcc -E main.c > main.tmp

How can this be achieved with CMake? The step should be applied on each c or cpp file of the project.

Answer 1

Use the first signature of add_custom_command to run the command to preprocess the file and then the command to do your custom additional processing, and define the OUTPUT as the file created by your custom processing, and then either list that file as a source file of the related target when defining it (via add_library/add_executable), or do it after defining it (via target_sources).

Working with compiler flags and definitions in CMake becomes a really tricky business if you want to take the approach that I will show. In general, you want to make sure the preprocessing step command has the exact same compiler definitions and flags as the compilation step command. Very strange, subtle, and possibly dangerous things can happen if the flags or definitions are different between preprocessing and compilation. You might get compiler errors, you might get linker errors, you might get runtime errors- all of which might be hard to understand-, or you might not get any errors and still have made a mistake. You will need to be extremely careful and know your compiler and inspect the preprocessor output to ensure that you've done things correctly.

There may be a better approach, but I haven't thought of it. I have never done this in a real project. The following answer is just theory. Treat it as a starting point. It is provided with no warranty (<insert the legal warranty statement you always see in open-source licenses here>). While I will make an effort to fix mistakes as they are found and keep this up-to-date, I will not accept responsibility if anything goes poorly as a result of using this.

Ex. for gcc (note: see here for docs on -E and .ii):

list(APPEND my_target_sources "${CMAKE_CURRENT_SOURCE_DIR}/file.cpp")

# this is best placed immediately after the creation of the target so that it knows all the directory compile options and such that the target saw when it got created.
foreach(input_file ${my_target_sources})
  set(output_file "${CMAKE_CURRENT_BINARY_DIR}/src-$<CONFIG>/file.ii")
  get_target_property(target_compile_opts my_target COMPILE_OPTIONS)
  add_custom_command(
    OUTPUT "${output_file}"
    COMMAND "${CMAKE_CXX_COMPILER}" ${CMAKE_CXX_FLAGS}  ${target_compile_opts} -o "${output_file}" -E "${input_file}"
    COMMAND my_additional_custom_processing_script "${output_file}" my_arg1 my_arg2 my_arg3
    DEPENDS "${input_file}" "my_additional_custom_processing_script"
    VERBATIM COMMAND_EXPAND_LISTS # these two args are generally useful, but not actually required here.
  )
  target_sources(my_target PRIVATE "${output_file}")

  # this might be needed since I don't expect cmake to infer CXX from the .ii extension
  set_property(SOURCE "${output_file}" TARGET_DIRECTORY my_target PROPERTY LANGUAGE CXX)
endforeach()

While the asker is using a python script, for the sake of generality to other future readers and an attempt to mark other very similar questions as duplicates, I have tried to leave that part a little generic, and wrote it as if the command could be found on the PATH by the host system, and could be run by itself (ie. if a script, assumes it has an appropriate shebang). Modify the custom preprocessing command part to suit your use-case.

If you want to set any compiler options on a per-source-file basis, you will need to make sure they get set in the custom command for that file, and also set them in the COMPILE_OPTIONS preporty for the post-preprocessed source file.

If anything about the pre-processing output depends on the general compiler flags that are config-specific (CMAKE_<LANG>_FLAGS_<CONFIG>), you will need to add those varibles wrapped in the right generator expression to the COMMAND, such as "$<$<CONFIG:Debug>:${CMAKE_CXX_FLAGS_DEBUG}>". I am not aware of a prettier way to do this for all configs.

This answer is currently missing:

• Setting INTERFACE compiler options from the dependencies of the target when preprocessing.
• Passing compile definitions associated with the target to the preprocessor step command.
  • Until I or someone finds out a good way to do this, you will probably immediately notice this when it comes to the presence or absence of the NDEBUG macro.

I really hope someone knows a better way to do this because my answer here is ripe with all kinds of opportunity for error.

Answer 2

Indeed, this had to be done manualy : I added a custom target where I generate the preprocessed files:

    add_custom_target(
    start_preprocessor 
    COMMAND make main.cpp.i 
    ..
    )

with another custom target, I make a backup of main.cpp and then rename main.cpp.i into main.cpp. I can then apply my preprocessing process. Afterwards, I restore the source files.