How to add external header files during bazel/tensorflow build

Question

I am trying to add external header file (like OpenCL header file) for some experimentation for tensorflow. I tried to add this into BUILD file under tensorflow/core/BUILD file:

# This includes implementations of all kernels built into TensorFlow.
cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/include"],    <==== this is the line I added

I have also created a softlink in this directory to the location of these header files from OpenCL driver (under tensorflow/third_party) too (like ln -s /opt/opencl/ ) but it still complains that it has not found that header file.

If I add external header file directly (like /opt/opencl/CL/) it complains that external files cannot be included (or some such thing).

I do not have root password to copy these header files into /usr/include/ too.

Can someone explain how exactly to do external header files into tensorflow for building?

Thanks for any quick help.

score 6 · Answer 1 · answered Nov 09 '16 at 10:07

I've faced with the similar problem when I built TensorFlow with Intel MKL and had to add MKL headers. My solution is the following:

Create symlink to your headers into third_party folder, like:

<your tensorflow folder>/third_party/opencl/include -> /opt/OpenCL/include

with command:

ln -s /opt/OpenCL/include <your tensorflow folder>/third_party/opencl

Create simple BUILD file into <your tensorflow folder>/third_party/opencl folder:

cc_library(
    name = "opencl",
    hdrs = glob(["include/CL/*.h"]),
    visibility = ["//visibility:public"],
)

Add deps into target library:

cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/opencl/include"],
    deps = [
        "//third_party/opencl", 
        ...
    ],
)

Don't forget to add compiler options either into target library as shown above or just as a flag to bazel:
```
 bazel build --copt="-Ithird_party/opencl/include" ...
```

If I'm elected president, my first executive order will be to have the `bazel` build project shut down, and the authors sent to a reeducation camp. — Zendel, Jan 09 '20 at 22:13

score 4 · Answer 2 · answered Jun 15 '16 at 19:40

You have to add the files as a dependency of this rule.

IIUC, you have the following structure:

tensorflow/
  BUILD
  WORKSPACE
  tensorflow/
    core/
      BUILD
  third_party/
    include -> /opt/opencl/CL # or something like that

You want to expose the .h files in a way Bazel can understand/depend on, so open up the tensorflow/BUILD file and add the following:

cc_library(
    name = "opencl",
    hdrs = glob(["third_party/include/*.h"]),
    visibility = ["//visibility:public"],
)

This creates a C++ library from the .h files under third_party/include, which can be depended on from anywhere in the source tree.

Now go to your tensorflow/core/BUILD file and add a dependency to the cc_library there:

cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/include"],
    deps = [
        "//:opencl", 
        # plus any other deps
    ],
)

Setting copts just changes the flags when gcc is run. Adding //:opencl to the dependencies tells Bazel to make those files available when gcc is run.

score 0 · Answer 3 · answered Jun 13 '16 at 23:37

0

Bazel tries to be very strict about making sure its builds only include files that it knows about, to try to make sure that they are reproducible. Unfortunately that can make it tough to experiment with. The correct way to solve the problem is to create a BUILD file and rule for the headers you want to include. You may be able to hack something by messing around with the bazel-* generated folders too, but I don't recommend it.

answered Jun 13 '16 at 23:37

Pete Warden

2,866
1
13
12

1

Thanks for the help, but I wanted to know how specifically do this. As I said in my original post, when I add the soft link to internal directory and give that as "-I" it still does not work. So, wanted to know exactly how we can do this. – Prakash Raghavendra Jun 15 '16 at 18:02

How to add external header files during bazel/tensorflow build

3 Answers3

Linked