Is it possible to accelerate clang-tidy using ccache or similar?

Question

Since employing ccache on our CI server, we find that the bottleneck in terms of build time is now our static analysis pass, that uses clang-tidy, among other tools. Does anyone know of a way to accelerate clang-tidy in a similar way to how ccache does so with a regular compiler?

Answer 1

I found another important detail here:

https://gitlab.kitware.com/cmake/cmake/-/merge_requests/1791/diffs

which is used here:

https://reviews.bitcoinabc.org/D5150?id=15995

So in order to be able to cache the output of the compiler when integrating clang-tidy using the : set(CMAKE_CXX_CLANG_TIDY ...

method you need to use the COMPILER_LAUNCHER method to configure ccache

find_program(CCACHE ccache)
if(CCACHE)
    set(CMAKE_C_COMPILER_LAUNCHER ${CCACHE})
    set(CMAKE_CXX_COMPILER_LAUNCHER ${CCACHE})
endif(CCACHE)

and NOT the launcher rule method:

find_program(CCACHE ccache)
if(CCACHE)
    set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ${CCACHE})
    set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ${CCACHE})
endif(CCACHE)

Answer 2

Thank you for the question and everybody for the answers.

After struggling to make ejfitzgerald/clang-tidy-cache working w/o any decent documentation or help I've given a try matus-chochlik/ctcache. It's indeed much easier in installation and configuration.

Here's a snippet we use to automatically use it with cmake when it's installed

find_program (CLANG_TIDY_CACHE_PATH NAMES "clang-tidy-cache")
if (CLANG_TIDY_CACHE_PATH)
    find_program (_CLANG_TIDY_PATH NAMES "clang-tidy" "clang-tidy-15" "clang-tidy-14" "clang-tidy-13" "clang-tidy-12")

    # Why do we use ';' here?
    # It's a cmake black magic: https://cmake.org/cmake/help/latest/prop_tgt/LANG_CLANG_TIDY.html#prop_tgt:%3CLANG%3E_CLANG_TIDY
    # The CLANG_TIDY_PATH is passed to CMAKE_CXX_CLANG_TIDY, which follows CXX_CLANG_TIDY syntax.
    set (CLANG_TIDY_PATH "${CLANG_TIDY_CACHE_PATH};${_CLANG_TIDY_PATH}" CACHE STRING "A combined command to run clang-tidy with caching wrapper")
else ()
    find_program (CLANG_TIDY_PATH NAMES "clang-tidy" "clang-tidy-15" "clang-tidy-14" "clang-tidy-13" "clang-tidy-12")
endif ()

set (CMAKE_CXX_CLANG_TIDY "${CLANG_TIDY_PATH}")

With it, if clang-tidy-cache is in the path, it will be used together with any found clang-tidy.

During the build, we define CTCACHE_DIR and preserve them between builds.

In our CI it caused reducing the linting time from 330 minutes to 22, so it's almost 15x faster!

Answer 3

There is clang-tidy-cache although i don't know how that works together with ccache.

Answer 4

I couldn't get clang-tidy-cache to build because I didn't have the correct go version, so I made cltcache in python instead. It is meant to be used exactly like ccache and is hopefully a lot easier to install than clang-tidy-cache.

It will hash the preprocessor output with comments along with the enabled checks and the clang-tidy-version string and use that as the key for the cache. Upon cache miss it will ä do the clang-tidy call and compress and store the stdout, the stderr and the return code in the cache.

Not sure if it is fast enough for other projects but in my project, clang-tidy consumes around 4x as much time as the actual compilation, so running a preprocessor an extra time consumes a comparatively negligible amount of time.

Answer 5

Finally, I have found a solution for this: it's switching the build system to bazel. bazel is a build system which is very generic and defines an action graph. Essentially every action has a set of inputs and outputs. Based on the inputs the outputs can be cached. Hence, bazel solves the problem at it's root.

The necessary rules for integrating clang-tidy in a cachable way can be found here: https://github.com/erenon/bazel_clang_tidy

In order to make use of caching you need to setup a remote cache. This can be done using a docker-compose script. The necessary docker container already exists. The command to get it running can be found here: https://github.com/buchgr/bazel-remote/

Finally, bazel also solves the problem of caching the result of the linking phase.

Not only that but bazel also allows building other languages like java. Hence, in complex projects it allows to replace all other build system with a single one.

Finally, bazel also allows parallelizing your build on a cluster.

Last but not least you can define several platforms and toolchains. All in all this allows to do cross-platform builds.