Dynamic dependency within custom source files

Question

I have custom source files that are not C/C++. These files are compiled to C source files and then those C files are compiled by cmake as usual.

However, my custom files can have dependencies that can change during development.

For example let us say the I have the files source1.lang, source2.lang, source3.lang...

source1.lang

... some code

source2.lang

import source1;
... some code

I compile these files as

./langc source1.lang
./langc source2.lang
...

and I get source1.c, source2.c ...

Now, as you can see source2.lang depends on source1.lang. I can find these dependencies for these custom files at any time by pre-processing them. And then generate an explicit dependency of source2.lang on source1.lang so that it gets recompiled if source1.lang is modified.

Right now, I have a custom command like this

# This is populated automatically.
set(SOURCE2_DEPENDS source2.lang source1.lang) 

add_custom_command(
        DEPENDS ${SOURCE2_DEPENDS }
        COMMAND ./langc source2.lang
        OUTPUT source2.c 
        COMMENT "Compiling source2.c"
     )

The problem is the dependencies can change during development. That is, SOURCE1_DEPENDS can change after configuration. For example I can modify source2.lang to import source3.lang

source2.lang

import source1;
import source3;
... some code

I can now see that source2.lang is modified. So I pre-process it to find the new dependencies. But how do I let cmake know that the dependency has changed?

The only way I can see so far is for Makefile generators to modify the generated build.make file to add and remove these dependencies manually.

However, how I can I tell cmake in a generic way that these are the new dependencies? Basically as cmake does normally when you add a new #include to a C source file. So that things work for other generators as well.

KamilCuk · Accepted Answer · 2023-08-31T06:35:31.090

4

The following I think only works as of now with Ninja generator. I think it also works with makefile generator, but I didn't check.

The add_custom_command has DEPFILE option. You have to generate a depfile in makefile-ish style from your command. For example:

add_custom_command(
        DEPENDS ${SOURCE2_DEPENDS}
        DEPFILE ${CMAKE_CURRENT_BINARY_DIR}/source2.d
        COMMAND ./langc 
               --depfile ${CMAKE_CURRENT_BINARY_DIR}/source2.d
               --depfile-relative ${CMAKE_BINARY_DIR}
                source2.lang
        OUTPUT source2.c 
        COMMENT "Compiling source2.c"
     )

The generated depfile should look like a makefile dependency file generated by gcc with the standard -MT/-MD options. Note the target path has to be relative to ${CMAKE_BINARY_DIR}. So langc program should generate the following file in ${CMAKE_CURRENT_BINARY_DIR}/source2.d:

buildir/subdir/source2.c: /absolute/path/to/be/safe/source2.lang /absolute/path/to/be/safe/source1.lang /etc..
# ^^ must be relative to CMAKE_BINARY_DIR

Cmake inserts depfile commands into generated ninja build system and they get picked by ninja and the generator then knows the depedency. But then, you could think about adding your own language to cmake so that *.lang files are picked automatically by cmake.

I did the DEPFILE generation on a project that I used m4 to preprocess source files. With --debug=p m4 option the preprocessor printed m4debug: including file <the file> or something like that while preprocessing the file. From those messages I extracted files and generated the dependency file. The code is available here m4.cmake and m4.sh.

edited Aug 31 '23 at 06:35

answered Jul 03 '20 at 10:41

KamilCuk

120,984
8
59
111

This is basically what I had in mind to do for Makefile generators. Basically manually inserting those `A:B` dependency rules to the end of the `build.make` file generated by cmake. – stardust Jul 03 '20 at 10:47
"But then, you could think about adding your own language to cmake so that *.lang files are picked automatically by cmake" How do I do that? That was something I was hoping would be possible. – stardust Jul 03 '20 at 10:48
I have never did it, so I have no experience. There is a stackoverflow thread somewhere that explains how to do it. – KamilCuk Jul 03 '20 at 10:52
I have looked for something like that over on the cmake documentation and on stackoverflow. No Luck. Can you please try to remember the thread if you can? It might help. – stardust Jul 03 '20 at 10:54
1

https://stackoverflow.com/questions/38293535/generic-rule-from-makefile-to-cmake The example there is a pearl. I think also inspecting existing files, like `CMakeDetermineCCompiler` `CMakeCCompiler.cmake.in` etc. in [cmake/Modules](https://github.com/Kitware/CMake/tree/master/Modules) will help. – KamilCuk Jul 03 '20 at 10:57
Thank you!. I hope that works on declaring dependency rules as well. Even if it turns out it does not work that is good to know anyway. – stardust Jul 03 '20 at 11:07
Based on the this answer I tried to create something for my situation but unfortunately I failed, see https://stackoverflow.com/questions/72518401/cmake-dependencies-on-included-files – albert Jun 11 '22 at 09:48
In my case, some of the m4_include calls didn't need the include path and so there was never any resolving to log with `--debug=p`. What I ended up using was `--trace=m4_include --debug=a --debugfile=${CMAKE_BINARY_DIR}/${target}.m4trace` – Daniel Schepler Aug 30 '23 at 21:33

Dynamic dependency within custom source files

1 Answers1

Linked