Bazel C++ precompiled headers implementation

Question

I have written an MSVC Precompiled Header Files (PCH) implementation for Bazel (2.0) and would like to get some feedback on it as I'm not happy with it.

To quickly recap what needs to be done to get PCH working in MSVC:

1. Compile the PCH with /Yc and /Fp to obtain the (1) .pch file and the (2) .obj file.
2. Compile the binary using the /Yu on (1) and again the same /Fp option.
3. Link the binary using the .obj file (2).

Implementation

We define a rule which takes the pchsrc (for /Yc) and pchhdr (for /Fp) as an argument as well as some of the cc_* rule arguments (to get the defines and includes). We then invoke the compiler to obtain the the PCH (mainly following the approach demonstrated here). Once we have the PCH, we propagate the location and linker inputs via CcInfo and the user needs to call cc_pch_copts to get the /Yu and /Fp options.

pch.bzl

load("@rules_cc//cc:action_names.bzl", "ACTION_NAMES")
load("@rules_cc//cc:find_cc_toolchain.bzl", "find_cc_toolchain")

def cc_pch_copts(pchheader, pchtarget):
  return [
    "/Yu\"" + pchheader + "\"", 
    "/Fp\"$(location :" + pchtarget + ")\""
  ]

def _cc_pch(ctx):
  """ Create a precompiled header """
  cc_toolchain = find_cc_toolchain(ctx)

  source_file = ctx.file.pchsrc
  pch_file = ctx.outputs.pch
  pch_obj_file = ctx.outputs.obj

  # Obtain the includes of the dependencies
  cc_infos = []
  for dep in ctx.attr.deps:
    if CcInfo in dep:
      cc_infos.append(dep[CcInfo])
  deps_cc_info = cc_common.merge_cc_infos(cc_infos=cc_infos)

  # Flags to create the pch
  pch_flags = [
    "/Fp" + pch_file.path, 
    "/Yc" + ctx.attr.pchhdr,  
  ]

  # Prepare the compiler
  feature_configuration = cc_common.configure_features(
    ctx = ctx,
    cc_toolchain = cc_toolchain,
    requested_features = ctx.features,
    unsupported_features = ctx.disabled_features,
  )

  cc_compiler_path = cc_common.get_tool_for_action(
    feature_configuration = feature_configuration,
    action_name = ACTION_NAMES.cpp_compile,
  )

  deps_ctx = deps_cc_info.compilation_context
  cc_compile_variables = cc_common.create_compile_variables(
    feature_configuration = feature_configuration,
    cc_toolchain = cc_toolchain,
    user_compile_flags = ctx.fragments.cpp.copts + ctx.fragments.cpp.cxxopts + pch_flags + ctx.attr.copts,
    source_file = source_file.path,
    output_file = pch_obj_file.path,
    preprocessor_defines = depset(deps_ctx.defines.to_list() + deps_ctx.local_defines.to_list() + ctx.attr.defines + ctx.attr.local_defines),
    include_directories = deps_ctx.includes,
    quote_include_directories = deps_ctx.quote_includes,
    system_include_directories = depset(["."] + deps_ctx.system_includes.to_list()),
    framework_include_directories = deps_ctx.framework_includes,
  )

  env = cc_common.get_environment_variables(
    feature_configuration = feature_configuration,
    action_name = ACTION_NAMES.cpp_compile,
    variables = cc_compile_variables,
  )

  command_line = cc_common.get_memory_inefficient_command_line(
    feature_configuration = feature_configuration,
    action_name = ACTION_NAMES.cpp_compile,
    variables = cc_compile_variables,
  )

  args = ctx.actions.args()
  for cmd in command_line:
    if cmd == "/showIncludes":
      continue
    args.add(cmd)

  # Invoke the compiler
  ctx.actions.run(
    executable = cc_compiler_path,
    arguments = [args],
    env = env,
    inputs = depset(
      items = [source_file],
      transitive = [cc_toolchain.all_files],
    ),
    outputs = [pch_file, pch_obj_file],
    progress_message = "Generating precompiled header {}".format(ctx.attr.pchhdr),
  )

  return [
    DefaultInfo(files = depset(items = [pch_file])),
    CcInfo(
      compilation_context=cc_common.create_compilation_context(
        includes=depset([pch_file.dirname]),
        headers=depset([pch_file]),
      ),
      linking_context=cc_common.create_linking_context(
        user_link_flags = [pch_obj_file.path]
      )
    )
  ]

cc_pch = rule(
  implementation = _cc_pch,
  attrs = {
    "pchsrc": attr.label(allow_single_file=True, mandatory=True),
    "pchhdr": attr.string(mandatory=True),
    "copts": attr.string_list(),
    "local_defines": attr.string_list(),
    "defines": attr.string_list(),
    "deps": attr.label_list(allow_files = True),
    "_cc_toolchain": attr.label(default = Label("@bazel_tools//tools/cpp:current_cc_toolchain")),
  },
  toolchains = ["@bazel_tools//tools/cpp:toolchain_type"],
  fragments = ["cpp"],
  outputs = {
    "pch": "%{pchsrc}.pch", 
    "obj": "%{pchsrc}.pch.obj"
  },
  provides = [CcInfo],
)

We would use it:

BUILD.bzl

load(":pch.bzl", "cc_pch", "cc_pch_copts")
load("@rules_cc//cc:defs.bzl", "cc_binary") 

def my_cc_binary(name, pchhdr, pchsrc, **kwargs):
  pchtarget = name + "_pch"
  cc_pch(
    name = pchtarget,
    pchsrc = pchsrc,
    pchhdr = pchhdr,
    defines = kwargs.get("defines", []),
    deps = kwargs.get("deps", []),
    local_defines = kwargs.get("local_defines", []),
    copts = kwargs.get("copts", []),
  )
  kwargs["deps"] = kwargs.get("deps", []) + [":" + pchtarget])
  kwargs["copts"] = kwargs.get("copts", []) + cc_pch_copts(pchhdr, pchtarget))

  native.cc_binary(name=name, **kwargs)

my_cc_binary(
  name = "main",
  srcs = ["main.cpp", "common.h", "common.cpp"],
  pchsrc = "common.cpp",
  pchhdr = "common.h",
)

with project being contained of:

main.cpp

#include "common.h"
int main() { std::cout << "Hello world!" << std::endl; return 0; }

common.h

#include <iostream>

common.cpp

#include "common.h"

Questions

The implementation works. However, my discussion points are:

• What is the best way to propagate the additional compile flags to dependent targets? The way I solved it via cc_pch_copts seems rather hacky. I would assume it involves defining a provider, but I couldn't find one which allows me to forward flags (CcToolChainConfigInfo has something in this direction but it seems overkill).
• Is there another way to get all the compile flags (defines, includes etc.) than what I implemented above? It's really verbose and it most doesn't cover a lot of corner cases. Would it be possible to do something like compiling an empty.cpp file in the cc_pch rule to obtain a provider which gives direct access to all the flags?

Note: I'm aware of the downsides of precompiled headers but this is a large codebase and not using it is unfortunately not an option.

Answer 1

Maybe it can be simplified by generating a dummy cpp just to trigger the generation of the pch file, there is no need to link the resulting obj. (like in qmake): You just define the name of the precomp header, it will generate a dummy precomp.h.cpp and use this to trigger the generation of the pch file.

In VS/msbuild it is also possible to just generate the pch from the precomp.h file (but requires change to the source): - change the item type of the header to "C/C++ compile" - set the /Yc option on this - add a hdrstop directive at the end of precomp.h like

#pragma once
#include <windows.h>
#pragma hdrstop("precomp.h") 

Thanks for sharing your bzl files, I'm also looking into this (large code base with precomp headers).

Answer 2

From what I know precompiled headers are especially usefull for framework developers doing lot of template metaprogramming and having a respectable code base. It is not intended to speed up the compilation if you are still in development of the framework. It does not speedup the compile time if the code is poorly designed and every dependencies comes in sequence. Your files here are only the config file of VC++, the actual job not even started yet and precompiled headers are bytecode.Use parallel build whenever possible.

Also, the resulting headers are HUGE !