CMake and FindProtobuf

Question

I'm using the FindProtobuf module in a project where the protocol buffer files are in a sub-directory. I want the CMakeLists.txt file in that sub-directory to invoke protoc to generate the CPP Files. My project folder structure is like this:

cammy/CMakeLists.txt   # Root CMakeLists.txt
cammy/protofiles/test.proto # protofile
cammy/protofiles/CMakeLists.txt

I have the include(FindProtobuf), the find_package invocation and the call to PROTOBUF_GENERATE_CPP in the CMakeLists.txt file in protobuf folder.

The executable build step is in the Root CMakeLists.txt file and I add the generated files to the target executable in this file ie.

add_executable( ${EXEC} ${SRCS} ${PROTO_SRC} ${PROTO_HEADER} )
target_link_libraries( ${EXEC} ${PROTOBUF_LIBRARIES} )

are both defined in the root CMakeLists.txt

When I run cmake, it does not run protoc to generate the Source files even though I expilicitly tie generated sources to the executable thereby creating a dependency.

When I move all the contents of CMakeLists.txt in the protofiles folder into the root CMakeLists.txt, the proto files are compiled.

Can anyone help me with this? I want all the protocol buffer building stuff to go in the CMakeLists.txt file created in the protofiles folder.

I also noticed that variables generated in the inner CMakeLists.txt ( like PROTO_SRC ) are defined in the inner file when printed ( ie I get the correct generated CPP filename ) but when I print the same variable in the root file.. it is empty. Its almost as though I need to "export" (if there were a way in cmake ) the variables out to the root folder.

Any help would be much appreciated.

Thanks Kartik

Answer 1

I think [FindProtobuf][0] isn't really meant to be used this way. From its docs:

NOTE: The PROTOBUF_GENERATE_CPP macro & add_executable() or add_library() calls only work properly within the same directory.

You're trying to use the PROTOBUF_GENERATE_CPP macro in a subdirectory, and although the CMake docs don't really make it clear, a subdirectory introduces a new scope for variables. This means that any variables set or modified in the subdir scope don't affect similarly-named variables in the parent scope. Hence the reason for PROTO_SRC being available in your protofiles dir, but not in the parent.

The way to pass variables up a scope is to use [set(... PARENT_SCOPE)][1], so in protofiles/CMakeLists.txt you could do:

PROTOBUF_GENERATE_CPP(PROTO_SRC PROTO_HEADER test.proto)

set(PROTOBUF_INCLUDE_DIRS ${PROTOBUF_INCLUDE_DIRS} PARENT_SCOPE)
set(PROTOBUF_LIBRARIES ${PROTOBUF_LIBRARIES} PARENT_SCOPE)
set(PROTO_SRC ${PROTO_SRC} PARENT_SCOPE)
set(PROTO_HEADER ${PROTO_HEADER} PARENT_SCOPE)

However, this still doesn't get us all the way!

CMake doesn't actually invoke the protoc compiler to generate the .pb.h and .pb.cc files - it uses [add_custom_command][2] to do this. The custom command specifies the .pb.h and .pb.cc files as outputs, and the custom command is only invoked (i.e. protoc executed) if a subsequent target which depends on these files is built.

So, at configure time (when CMake executes) these files don't exist. This is a problem if you try to add them as sources to an add_library or add_executable command - CMake needs to be told that these files don't exist when it runs, but that they will exist at build time.

The way to do that is to set the [GENERATED][3] property to TRUE for these files. The PROTOBUF_GENERATE_CPP macro does that automatically, but as with the variables, the property isn't populated up into the parent scope. So in your top-level CMakeLists.txt, you also need to add:

set_source_files_properties(${PROTO_SRC} ${PROTO_HEADER} PROPERTIES
                            GENERATED TRUE)

As you can see, using PROTOBUF_GENERATE_CPP in a different dir to the corresponding add_library/add_executable commands is a bit fragile. If you can avoid doing it, you probably should.

[0]: https://cmake.org/cmake/help/latest/module/FindProtobuf.html "CMake latest documentation for "FindProtobuf" module" [1]: https://cmake.org/cmake/help/latest/command/set.html "CMake latest documentation for "set" command" [2]: https://cmake.org/cmake/help/latest/command/add_custom_command.html "CMake latest documentation for "add_custom_command"" [3]: https://cmake.org/cmake/help/latest/prop_sf/GENERATED.html "CMake latest documentation for "GENERATED" source file property"