2

I wonder which headers are implicitely included in the .cu compilation unit. For instance, let's take some basic example:

#include <iostream>

__global__ void kernel(int a, int b, int *c)
{
    *c = a * b;
}

int main()
{
    int c, *dev_c;

    cudaMalloc((void **) &dev_c, sizeof(*dev_c));
    kernel<<<1,1>>>(5, 3, dev_c);
    cudaMemcpy(&c, dev_c, sizeof(*dev_c), cudaMemcpyDeviceToHost);
    cudaFree(dev_c);

    std::cout << c << std::endl;
}

According to the C++ Standard all functions must be declared prior to their usage (likewise in C since C99), so I guess that these CUDA headers are included "behind the scence".

The question is relevant as I would like to get IntelliSense within Visual C++ for CUDA API invocations and I haven't found any "oficial" reference on this.

Grzegorz Szpetkowski
  • 36,988
  • 6
  • 90
  • 137

1 Answers1

4

I ran the following command for an empty file on Linux:

nvcc --generate-dependencies empty.cu

This resulted in the following list:

empty.o : empty.cu \
/usr/include/stdc-predef.h \
/opt/cuda/bin/..//include/cuda_runtime.h \
/opt/cuda/bin/..//include/host_config.h \
/usr/include/features.h \
/usr/include/x86_64-linux-gnu/sys/cdefs.h \
/usr/include/x86_64-linux-gnu/bits/wordsize.h \
/usr/include/x86_64-linux-gnu/gnu/stubs.h \
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
/opt/cuda/bin/..//include/builtin_types.h \
/opt/cuda/bin/..//include/device_types.h \
/opt/cuda/bin/..//include/host_defines.h \
/opt/cuda/bin/..//include/driver_types.h \
/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/limits.h \
/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/syslimits.h \
/usr/include/limits.h \
/usr/include/x86_64-linux-gnu/bits/posix1_lim.h \
/usr/include/x86_64-linux-gnu/bits/local_lim.h \
/usr/include/linux/limits.h \
/usr/include/x86_64-linux-gnu/bits/posix2_lim.h \
/usr/include/x86_64-linux-gnu/bits/xopen_lim.h \
/usr/include/x86_64-linux-gnu/bits/stdio_lim.h \
/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h \
/opt/cuda/bin/..//include/surface_types.h \
/opt/cuda/bin/..//include/texture_types.h \
/opt/cuda/bin/..//include/vector_types.h \
/opt/cuda/bin/..//include/channel_descriptor.h \
/opt/cuda/bin/..//include/cuda_runtime_api.h \
/opt/cuda/bin/..//include/cuda_device_runtime_api.h \
/opt/cuda/bin/..//include/driver_functions.h \
/opt/cuda/bin/..//include/vector_functions.h \
/opt/cuda/bin/..//include/vector_functions.hpp \
/opt/cuda/bin/..//include/common_functions.h \
/usr/include/string.h \
/usr/include/xlocale.h \
/usr/include/time.h \
/usr/include/x86_64-linux-gnu/bits/time.h \
/usr/include/x86_64-linux-gnu/bits/types.h \
/usr/include/x86_64-linux-gnu/bits/typesizes.h \
/usr/include/x86_64-linux-gnu/bits/timex.h \
/usr/include/c++/4.8/new \
/usr/include/x86_64-linux-gnu/c++/4.8/bits/c++config.h \
/usr/include/x86_64-linux-gnu/c++/4.8/bits/os_defines.h \
/usr/include/x86_64-linux-gnu/c++/4.8/bits/cpu_defines.h \
/usr/include/c++/4.8/exception \
/usr/include/c++/4.8/bits/atomic_lockfree_defines.h \
/usr/include/stdio.h \
/usr/include/libio.h \
/usr/include/_G_config.h \
/usr/include/wchar.h \
/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h \
/usr/include/x86_64-linux-gnu/bits/sys_errlist.h \
/usr/include/stdlib.h \
/usr/include/x86_64-linux-gnu/bits/waitflags.h \
/usr/include/x86_64-linux-gnu/bits/waitstatus.h \
/usr/include/endian.h \
/usr/include/x86_64-linux-gnu/bits/endian.h \
/usr/include/x86_64-linux-gnu/bits/byteswap.h \
/usr/include/x86_64-linux-gnu/bits/byteswap-16.h \
/usr/include/x86_64-linux-gnu/sys/types.h \
/usr/include/x86_64-linux-gnu/sys/select.h \
/usr/include/x86_64-linux-gnu/bits/select.h \
/usr/include/x86_64-linux-gnu/bits/sigset.h \
/usr/include/x86_64-linux-gnu/sys/sysmacros.h \
/usr/include/x86_64-linux-gnu/bits/pthreadtypes.h \
/usr/include/alloca.h \
/usr/include/x86_64-linux-gnu/bits/stdlib-float.h \
/usr/include/assert.h \
/opt/cuda/bin/..//include/math_functions.h \
/usr/include/math.h \
/usr/include/x86_64-linux-gnu/bits/huge_val.h \
/usr/include/x86_64-linux-gnu/bits/huge_valf.h \
/usr/include/x86_64-linux-gnu/bits/huge_vall.h \
/usr/include/x86_64-linux-gnu/bits/inf.h \
/usr/include/x86_64-linux-gnu/bits/nan.h \
/usr/include/x86_64-linux-gnu/bits/mathdef.h \
/usr/include/x86_64-linux-gnu/bits/mathcalls.h \
/usr/include/c++/4.8/cmath \
/usr/include/c++/4.8/bits/cpp_type_traits.h \
/usr/include/c++/4.8/ext/type_traits.h \
/usr/include/c++/4.8/cstdlib \
/opt/cuda/bin/..//include/math_functions.hpp \
/opt/cuda/bin/..//include/math_functions_dbl_ptx3.h \
/opt/cuda/bin/..//include/math_functions_dbl_ptx3.hpp \
/opt/cuda/bin/..//include/cuda_surface_types.h \
/opt/cuda/bin/..//include/cuda_texture_types.h \
/opt/cuda/bin/..//include/device_functions.h \
/opt/cuda/bin/..//include/device_functions.hpp \
/opt/cuda/bin/..//include/device_atomic_functions.h \
/opt/cuda/bin/..//include/device_atomic_functions.hpp \
/opt/cuda/bin/..//include/device_double_functions.h \
/opt/cuda/bin/..//include/device_double_functions.hpp \
/opt/cuda/bin/..//include/sm_20_atomic_functions.h \
/opt/cuda/bin/..//include/sm_20_atomic_functions.hpp \
/opt/cuda/bin/..//include/sm_32_atomic_functions.h \
/opt/cuda/bin/..//include/sm_32_atomic_functions.hpp \
/opt/cuda/bin/..//include/sm_35_atomic_functions.h \
/opt/cuda/bin/..//include/sm_20_intrinsics.h \
/opt/cuda/bin/..//include/sm_20_intrinsics.hpp \
/opt/cuda/bin/..//include/sm_30_intrinsics.h \
/opt/cuda/bin/..//include/sm_30_intrinsics.hpp \
/opt/cuda/bin/..//include/sm_32_intrinsics.h \
/opt/cuda/bin/..//include/sm_32_intrinsics.hpp \
/opt/cuda/bin/..//include/sm_35_intrinsics.h \
/opt/cuda/bin/..//include/surface_functions.h \
/opt/cuda/bin/..//include/surface_functions.hpp \
/opt/cuda/bin/..//include/texture_fetch_functions.h \
/opt/cuda/bin/..//include/texture_fetch_functions.hpp \
/opt/cuda/bin/..//include/texture_indirect_functions.h \
/opt/cuda/bin/..//include/texture_indirect_functions.hpp \
/opt/cuda/bin/..//include/surface_indirect_functions.h \
/opt/cuda/bin/..//include/surface_indirect_functions.hpp \
/opt/cuda/bin/..//include/device_launch_parameters.h

So nvcc by default already includes a lot of CUDA headers (in my case at /opt/cuda/...).

I then created a file test.cpp with the following content and ran g++ -H -D__CUDACC__ (to fake nvcc):

#include "/opt/cuda/bin/..//include/cuda_runtime.h"

This resulted in the following list:

. /opt/cuda/bin/..//include/cuda_runtime.h
.. /opt/cuda/bin/..//include/host_config.h
... /usr/include/features.h
.... /usr/include/x86_64-linux-gnu/sys/cdefs.h
..... /usr/include/x86_64-linux-gnu/bits/wordsize.h
.... /usr/include/x86_64-linux-gnu/gnu/stubs.h
..... /usr/include/x86_64-linux-gnu/gnu/stubs-64.h
.. /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/device_types.h
.... /opt/cuda/bin/..//include/host_defines.h
... /opt/cuda/bin/..//include/driver_types.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/limits.h
..... /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/syslimits.h
...... /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/limits.h
....... /usr/include/limits.h
........ /usr/include/x86_64-linux-gnu/bits/posix1_lim.h
......... /usr/include/x86_64-linux-gnu/bits/local_lim.h
.......... /usr/include/linux/limits.h
........ /usr/include/x86_64-linux-gnu/bits/posix2_lim.h
........ /usr/include/x86_64-linux-gnu/bits/xopen_lim.h
......... /usr/include/x86_64-linux-gnu/bits/stdio_lim.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
... /opt/cuda/bin/..//include/surface_types.h
... /opt/cuda/bin/..//include/texture_types.h
... /opt/cuda/bin/..//include/vector_types.h
.... /opt/cuda/bin/..//include/builtin_types.h
..... /opt/cuda/bin/..//include/vector_types.h
.. /opt/cuda/bin/..//include/channel_descriptor.h
... /opt/cuda/bin/..//include/cuda_runtime_api.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/cuda_device_runtime_api.h
.. /opt/cuda/bin/..//include/driver_functions.h
... /opt/cuda/bin/..//include/builtin_types.h
.. /opt/cuda/bin/..//include/vector_functions.h
... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/vector_functions.hpp
.... /opt/cuda/bin/..//include/builtin_types.h
.. /opt/cuda/bin/..//include/common_functions.h
... /opt/cuda/bin/..//include/builtin_types.h
... /usr/include/string.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
.... /usr/include/xlocale.h
... /usr/include/time.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
.... /usr/include/x86_64-linux-gnu/bits/time.h
..... /usr/include/x86_64-linux-gnu/bits/types.h
...... /usr/include/x86_64-linux-gnu/bits/wordsize.h
...... /usr/include/x86_64-linux-gnu/bits/typesizes.h
..... /usr/include/x86_64-linux-gnu/bits/timex.h
... /opt/cuda/bin/..//include/math_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /usr/include/math.h
..... /usr/include/x86_64-linux-gnu/bits/huge_val.h
..... /usr/include/x86_64-linux-gnu/bits/huge_valf.h
..... /usr/include/x86_64-linux-gnu/bits/huge_vall.h
..... /usr/include/x86_64-linux-gnu/bits/inf.h
..... /usr/include/x86_64-linux-gnu/bits/nan.h
..... /usr/include/x86_64-linux-gnu/bits/mathdef.h
..... /usr/include/x86_64-linux-gnu/bits/mathcalls.h
..... /usr/include/x86_64-linux-gnu/bits/mathcalls.h
..... /usr/include/x86_64-linux-gnu/bits/mathcalls.h
.... /usr/include/stdlib.h
..... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
..... /usr/include/x86_64-linux-gnu/bits/waitflags.h
..... /usr/include/x86_64-linux-gnu/bits/waitstatus.h
...... /usr/include/endian.h
....... /usr/include/x86_64-linux-gnu/bits/endian.h
....... /usr/include/x86_64-linux-gnu/bits/byteswap.h
........ /usr/include/x86_64-linux-gnu/bits/wordsize.h
........ /usr/include/x86_64-linux-gnu/bits/byteswap-16.h
..... /usr/include/x86_64-linux-gnu/sys/types.h
...... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
...... /usr/include/x86_64-linux-gnu/sys/select.h
....... /usr/include/x86_64-linux-gnu/bits/select.h
........ /usr/include/x86_64-linux-gnu/bits/wordsize.h
....... /usr/include/x86_64-linux-gnu/bits/sigset.h
....... /usr/include/x86_64-linux-gnu/bits/time.h
...... /usr/include/x86_64-linux-gnu/sys/sysmacros.h
...... /usr/include/x86_64-linux-gnu/bits/pthreadtypes.h
....... /usr/include/x86_64-linux-gnu/bits/wordsize.h
..... /usr/include/alloca.h
...... /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h
..... /usr/include/x86_64-linux-gnu/bits/stdlib-float.h
.... /usr/include/c++/4.8/cmath
..... /usr/include/x86_64-linux-gnu/c++/4.8/bits/c++config.h
...... /usr/include/x86_64-linux-gnu/c++/4.8/bits/os_defines.h
...... /usr/include/x86_64-linux-gnu/c++/4.8/bits/cpu_defines.h
..... /usr/include/c++/4.8/bits/cpp_type_traits.h
..... /usr/include/c++/4.8/ext/type_traits.h
.... /usr/include/c++/4.8/cstdlib
.... /opt/cuda/bin/..//include/math_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/math_functions_dbl_ptx3.h
..... /opt/cuda/bin/..//include/math_functions_dbl_ptx3.hpp
.. /opt/cuda/bin/..//include/cuda_surface_types.h
... /opt/cuda/bin/..//include/builtin_types.h
.. /opt/cuda/bin/..//include/cuda_texture_types.h
... /opt/cuda/bin/..//include/builtin_types.h
.. /opt/cuda/bin/..//include/device_functions.h
... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/device_functions.hpp
.... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/device_atomic_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/device_atomic_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/device_double_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/device_double_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_20_atomic_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/sm_20_atomic_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_32_atomic_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/sm_32_atomic_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_35_atomic_functions.h
... /opt/cuda/bin/..//include/sm_20_intrinsics.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/sm_20_intrinsics.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_30_intrinsics.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/sm_30_intrinsics.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_32_intrinsics.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/sm_32_intrinsics.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/sm_35_intrinsics.h
... /opt/cuda/bin/..//include/surface_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/surface_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/texture_fetch_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/texture_fetch_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/texture_indirect_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/texture_indirect_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
... /opt/cuda/bin/..//include/surface_indirect_functions.h
.... /opt/cuda/bin/..//include/builtin_types.h
.... /opt/cuda/bin/..//include/surface_indirect_functions.hpp
..... /opt/cuda/bin/..//include/builtin_types.h
.. /opt/cuda/bin/..//include/device_launch_parameters.h

Looks like cuda_runtime.h includes all other CUDA related headers.

m.s.
  • 16,063
  • 7
  • 53
  • 88
  • Would it possible that `cuda_runtime.h` is "top header" for these below it? I mean if ordering of `#include` directives is the same as above, then it might make some sense to assume that. In case of `/usr/include/stdc-predef.h` I believe that it's some kind of different story, it was added in GCC, since version 4.8. – Grzegorz Szpetkowski Apr 17 '15 at 22:56
  • Unfortunately, nvcc does not support the `-H` flag, otherwise we could print a [hierachy of included files](http://stackoverflow.com/a/6685693/678093). Yes, I think the headers from `/usr/include/` come from GCC. – m.s. Apr 17 '15 at 23:01
  • Thanks, I think that second list might be different because of conditional `#include` directives, depending on various settings. For instance my copy of `host_config.h` checks if `__CUDACC__` macro is defined. If not, then it skipping some parts. There might be likely more than that I suppose. – Grzegorz Szpetkowski Apr 17 '15 at 23:24
  • @GrzegorzSzpetkowski: You are right, I now added `-D__CUDACC__` and got a complete looking list. – m.s. Apr 17 '15 at 23:29