Check if a ccache call was a cache hit

Question

As part of my build process, I'd like to get statistics on the build time and whether ccache found the item in the cache. I know about ccache -s where I can compare the previous and current cache hit counts.

However, if I have hundreds of compilation threads running in parallel, the statistics don't tell me which file caused the hit.

The return code of ccache is that of the compiler. Is there any way I can get ccache to tell me if it was successful?

Answer 1

There are two options:

1. Enable the ccache log file: Set log_file in the configuration (or the environment variable CCACHE_LOGFILE) to a file path. Then you can figure out the result of each compilation from the log data. It can be a bit tedious if there are many parallel ccache invocations (the log file is shared between all of them, so log records from the different processes will be interleaved) but possible by taking the PID part of each log line into account.
2. In ccache 3.5 and newer, it's better to enable the debug mode: Set debug = true in the configuration (or the environment variable CCACHE_DEBUG=1). ccache will then store the log for each produced object file in <objectfile>.ccache-log. Read more in Cache debugging in the ccache manual.

Answer 2

I wrote a quick-n-dirty script that tells me which files had to be rebuild and what the cache miss ratio was:

Sample output (truncated):

ccache hit: lib/expression/unary_minus_expression.cpp
ccache miss: lib/expression/in_expression.cpp
ccache miss: lib/expression/arithmetic_expression.cpp

=== 249 files, 248 cache misses (0.995984 %)===

Script:

#!/usr/bin/env python3

from pathlib import Path
import re
import os

files = {}
for filename in Path('src').rglob('*.ccache-log'):
  with open(filename, 'r') as file:
    for line in file:
      source_file_match = re.findall(r'Source file: (.*)', line)
      if source_file_match:
        source_file = source_file_match[0]
      result_match = re.findall(r'Result: cache (.*)', line)
      if result_match:
        result = result_match[0]
        files[source_file] = result
        break

if len(files) == 0:
  print("No *.ccache-log files found. Did you compile with ccache and the environment variable CCACHE_DEBUG=1?")
  sys.exit(1)

common_path_prefix = os.path.commonprefix(list(files.keys()))
files_shortened = {}

misses = 0
for file in files:
  shortened = file.replace(common_path_prefix, '')
  if files[file] == 'miss':
    misses += 1
    print("ccache miss: %s" % (shortened))

print("\n=== %i files, %i cache misses (%f %%)===\n" % (len(files), misses, float(misses) / len(files) * 100))

Note that this takes all ccache-log files into account, not only those of the last build. If you want the latter, simply remove the log files first.