1

I have Tensorflow 2.2 and Cuda 10.1 with cuDnn 8.0.3

I am unable to run my scripts because it keeps looking for cuDnn 7 dll file: cudnn64_7.dll I get the following:

Could not load dynamic library 'cudnn64_7.dll'; dlerror: cudnn64_7.dll not found

Even though I installed the newly published cuDnn 8.0.3 for Cuda 10.1 (see cuDNN 8.x support matrix) I went back to cuDNN 7.6.5 but I was hoping to get the "5 times faster" cuDNN v8.0 as NVIDIA claims.

Any help or workarounds on how to get this done? Googling gets me literally less than 5 results! as it seems not many got to try the new 8.0.3 (the one for 10.1)

talonmies
  • 70,661
  • 34
  • 192
  • 269
salouri
  • 760
  • 1
  • 8
  • 19
  • when a binary (like Tensorflow) is compiled, it is linked against various libraries. In the case of cudnn (and cudart, and other CUDA libraries) the specific version (within some bounds) that the binary is linked against is the version you must provide. You cannot substitute `cudnn64_8.dll` if the TF binary was linked against `cudnn64_7.dll`. In order to make that substitution work, you would have to recompile/rebuild the TF binaries. It's evident, from the error message, that your particular TF binaries are expecting a 7.x version of cudnn. – Robert Crovella Sep 15 '20 at 21:44
  • I looked at the lib files of my TF project and replaced all 64_7 to 64_8 names/configs. Yet I got nowhere with that. I wonder if compiling TF from source would help (not supported on windows 10 though) – salouri Sep 16 '20 at 06:34
  • same issue.. Installed CUDA 10.1 + cuDnn 8.0.3 and same error – Alex Nov 17 '20 at 11:42

1 Answers1

2

Had the same issue. 8.0.3 version is the current and latest supported version of the library for CUDA 10.1. However, tensorflow is build for the earlier version, so you have to use that instead.

To elaborate, if you check this page: https://www.tensorflow.org/install/source_windows#tested_build_configurations

+----------------------+----------------+-----------+-------------+-------+------+
|       Version        | Python version | Compiler  | Build tools | cuDNN | CUDA |
+----------------------+----------------+-----------+-------------+-------+------+
| tensorflow_gpu-2.3.0 | 3.5-3.8        | MSVC 2019 | Bazel 3.1.0 |   7.6 | 10.1 |
+----------------------+----------------+-----------+-------------+-------+------+

so, unless you build the TF locally - you have to use the supported version of cudnn.

That being said, however, if you check latest TF releases: https://github.com/tensorflow/tensorflow/releases

you will then see the following TensorFlow 2.4.0-rc1 note:

TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

You can use the release candidate version of TF, but then, you also have to upgrade CUDA to 11 (I am guessing version 11.0 since no postfix is mentioned) and use the cuDNN v8.0.2 (July 24th, 2020), for CUDA 11.0.

Just tested - this setup works. You just have to make sure to install numpy version 1.19.3 in order to avoid the problem mentioned in these threads

RuntimeError: The current Numpy installation fails to pass a sanity check due to a bug in the windows runtime

https://developercommunity.visualstudio.com/content/problem/1207405/fmod-after-an-update-to-windows-2004-is-causing-a.html

Alex
  • 4,607
  • 9
  • 61
  • 99
  • I accepted the answer even though I can't test it. Based on my experience and the TF team (https://github.com/tensorflow/tensorflow/issues/41892#issuecomment-667452483) TF2.3 and above won't support 5.0 compute capability of my GPU. I tried to find a reference for any change on this but didn't find any... – salouri Jan 26 '21 at 15:52