detect cuda/GPU during runtime

Question

I'm writing a shared library containing cuda code, however cuda related code is only a very small part of that library and in most use cases users don't even need that cuda related features. For me, I don't want to compile and publish two versions: CPU version and GPU version. So I want to detect if cuda/GPU environment is available in users' runtime regardless they have a graphic card or not, is it possible? I've searched for a few answers but it's still unclear to me what's the minimal requirement for the library users to "detect cuda" in runtime?

Since I'm publishing a shared library, which dynamic linking to cuda, so I suppose users need to at least install cuda toolkit on his machine regardless he got a GPU or not? otherwise, my library will fail at the very beginning when trying to find cuda.so? Is my understanding correct....? And what would be the best practice to publish a library compatible with both GPU/non-GPU environments and decide the behavior during runtime?

Answer 1

For applications that use the CUDA runtime library/API:

1. Statically link your app or shared library against libcudart_static.a (instead of libcudart.so). When compiling with nvcc, this is automatic, but you can specify -cudart static if you wish. This won't trigger an implicit dynamic link against libcuda.so If not using nvcc, then -lcudart_static is the link request (e.g. for ld).

2. Early in your program/library, do an "innocuous" CUDA call, such as cudaGetDevice(). If you get an error code, it is safe to assume that CUDA is non-functional on that system, and you should not attempt to use it.

This method doesn't require the CUDA toolkit to be installed on the target machine. Assuming you only use the CUDA runtime library, then that is all that is needed. If the machine has a proper GPU and driver install whose version is sufficient for the version of libcudart statically linked to your library, it will just work.

If your application uses other CUDA libraries such as CUBLAS, etc. then these would also need to be statically linked to your app/library.

This answer has an example, although it primarily has CUFFT in view.