13

I've the following error. I'm using a conda installation of tensorflow. I'm struggling to try to use it with my GPU.

Loaded runtime CuDNN library: 5005 (compatibility version 5000) but source was compiled with 5103 (compatibility version 5100). If using a binary install, upgrade your CuDNN library to match. If building from sources, make sure the library loaded at runtime matches a compatible version specified during compile configuration. F tensorflow/core/kernels/conv_ops.cc:526] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms) Aborted (core dumped)

which nvcc returns /usr/local/cuda-7.5/bin/nvcc

nvcc version returns Cuda compilation tools, release 7.5, V7.5.17

I tried downloading CuDNN v5.1 and did the following but it didn't work either ``` sudo cp lib* /usr/local/cuda-7.5/lib64/ sudo cp include/cudnn.h /usr/local/cuda-7.5/include/ sudo ldconfig

```

I tried on the other folder too sudo cp lib* /usr/local/cuda/lib64/ sudo cp include/cudnn.h /usr/local/cuda/include/ sudo ldconfig

Ritchie
  • 355
  • 2
  • 6
  • 17
  • Could you use LD_DEBUG to find which version of the cudnn library is loaded, and the location of that library? If it is still the old library, you can update it with the v5.1. – Yao Zhang Oct 05 '16 at 00:27
  • Same problem here - any luck so far? I tried using LD_DEBUG, but I must be doing something wrong. I run `LD_DEBUG=all cat` and then in another window I run the TensorFlow code, but nothing shows up. – chris Oct 11 '16 at 04:08
  • This is a major issue for me. I had to make sure I reinstalled everything. Are you running on AWS or your local computer? – Ritchie Oct 11 '16 at 22:30
  • Is there an answer to this problem? – Dror Hilman Nov 28 '16 at 06:41
  • I wish someone could explain to me what the error even means. – Charlie Parker Dec 06 '16 at 20:41
  • I made a question to help decipher what the error even means to make some progress on this error: http://stackoverflow.com/questions/41005249/what-does-the-error-loaded-runtime-cudnn-library-5005-but-source-was-compiled – Charlie Parker Dec 06 '16 at 21:29
  • did you try using different versions of cuda and cudnn? I tried cuda 7.5 and cudnn 5.1-8.0 and it worked (what I mean with 5.1-8.0 is that in the cluter I am using the system I believe figures out by itself which one to use and when I was using 5.0 alone it didn't work but now with that dash it works...I think I tried 8.0 for cuda and it didn't work. I am using 0.11.0 for tensorflow). Hope this helps and gives you a place to start off (also my other question has an answer might might be helpful to you or future readers of this question). – Charlie Parker Dec 07 '16 at 16:25

1 Answers1

14

There's a good explanation of what it means here - What does the error: `Loaded runtime CuDNN library: 5005 but source was compiled with 5103` mean?

Short answer is that you have CuDNN 5.0 but you should install CuDNN 5.1

It looks like that's what you are attempting to do. It worked for me to just follow the the instructions here - https://www.tensorflow.org/get_started/os_setup#optional_install_cuda_gpus_on_linux

Before doing this, I checked the contents of /usr/local/cuda/include/cudnn.h and indeed it had these lines towards the top indicating it was version 5.0.5

#define CUDNN_MAJOR      5
#define CUDNN_MINOR      0
#define CUDNN_PATCHLEVEL 5

If your /usr/local/cuda/include/cudnn.h is already 5.1, then there is another CuDNN in another directory that is getting referenced. I have the following in my .bashrc -- perhaps try adding this or checking the Tensorflow instructions for what to add.

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
export CUDA_HOME="/usr/local/cuda"
Community
  • 1
  • 1
Melinda Weathers
  • 2,649
  • 25
  • 31