2

Somehow my CUDA binary build process has been messed up. All of the .cu files compile nicely to .o files, but when I try to link, I get:

CMakeFiles/tester.dir/tester_intermediate_link.o: In function `__cudaRegisterLinkedBinary_66_tmpxft_00007a5f_00000000_16_cuda_device_runtime_compute_52_cpp1_ii_8b1a5d37':
/tmp/tmpxft_00006b54_00000000-2_tester_intermediate_link.reg.c:7: undefined reference to `__fatbinwrap_66_tmpxft_00007a5f_00000000_16_cuda_device_runtime_compute_52_cpp1_ii_8b1a5d37'

Now, I have not used compute_52 anywhere. My nvcc command-line is:

/usr/local/cuda/bin/nvcc -M -D__CUDACC__ /home/joeuser/src/my_project/src/kernel_specific/elementwise/Add.cu -o /home/joeuser/src/my_project/CMakeFiles/tester.dir/src/kernel_specific/elementwise/tester_generated_Add.cu.o.NVCC-depend -ccbin /usr/bin/gcc-4.9.3 -m64 --std c++11 -D__STRICT_ANSI__ -Xcompiler ,\"-Wall\",\"-g\",\"-g\",\"-O0\" -gencode arch=compute_35,code=compute_35 -g -G --generate-line-info -DNVCC -I/usr/local/cuda/include -I/opt/cub -I/usr/local/cuda/include

and my link line is:

/usr/bin/g++-4.9.3   -Wall -std=c++11 -g   some.o files.o here.o blah.o blahblah.o bar.cu.o baz.cu.o -o bin/myapp -rdynamic -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -lrt -ldl /usr/lib/libboost_system.so /usr/lib/libboost_program_options.so -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -lrt -ldl /usr/local/cuda/extras/CUPTI/lib64/libcupti.so -lnvToolsExt -lOpenCL /usr/lib/libboost_system.so /usr/lib/libboost_program_options.so /usr/local/cuda/extras/CUPTI/lib64/libcupti.so -lnvToolsExt -lOpenCL -Wl,-rpath,/usr/lib:/usr/local/cuda/extras/CUPTI/lib64

I'll note I have separate compilation enabled, and do not seem to have skipped my intermediate link phase.

Why is this happening?

talonmies
  • 70,661
  • 34
  • 192
  • 269
einpoklum
  • 118,144
  • 57
  • 340
  • 684
  • 2
    I would guess you have separate compilation enabled somewhere and have skipped a device link stage of the cuda device runtime library. But you haven't really provided a serious illustration of the problem, so who knows..... – talonmies Feb 29 '16 at 16:18

2 Answers2

2

CUDA has two compilation modes, relocatable and static.
The relocatable mode is required for some configurations-which we will not get into now.

If you want to compile in relocatable mode -rdc=true, you'll need the Cuda device runtime library.
Which is located in the file cudadevrt.lib.
On some instances, supplying -lcudadevrt as a command line switch to the CUDA linker does the job, but on e.g. MSVC, you'll also need to specify cudadebrt.lib as a link dependency.

enter image description here

Johan
  • 74,508
  • 24
  • 191
  • 319
  • 1
    Even though I had asked the question about a normal (i.e. not Windows) system, this complements the answer and the explanation is also useful. Thank you. – einpoklum Nov 27 '19 at 11:13
1

Well, I'm not sure why I'm seeing missing references to Compute 5.2 calls, but adding -lcudadevrt to the end of the link command makes the error go away.

talonmies
  • 70,661
  • 34
  • 192
  • 269
einpoklum
  • 118,144
  • 57
  • 340
  • 684