Which TensorFlow and CUDA version combinations are compatible?

Question

I have noticed that some newer TensorFlow versions are incompatible with older CUDA and cuDNN versions. Does an overview of the compatible versions or even a list of officially tested combinations exist? I can't find it in the TensorFlow documentation.

The question was addressing compatibility and (officially) tested combinations which, in my view, are not provided in the instructions for installation. Also, I cannot find the section you're referring to. These observations result in my overall view that the requested information is hard to find and therefore justifies providing easy access to the link posted in the answer. — whiletrue, May 31 '18 at 11:44
You will find that the CUDA and cuDNN versions on the page you mention match the one of the installation instructions. — P-Gn, May 31 '18 at 11:46
To find the installation instructions, go to the page I linked above then follow the link for your OS. — P-Gn, May 31 '18 at 11:47
The section you're referring to just gives me the compatible version for CUDA and cuDNN --ONCE-- I have found out about my desired TensorFlow version. In the common case (for example in .edu lab environments) where CUDA and cuDNN are already installed but TF not, the necessity for an overview becomes apparent. My claim is that it's quite hidden and hard to find (I wasn't apple to find it via googling). — whiletrue, May 31 '18 at 12:00
Oh I see what you mean -- trying to see which tensorflow version fits a particular CUDA/cuDNN combination. You could browse TF's release notes but the table you link to is indeed a good summary. — P-Gn, May 31 '18 at 13:15
Working : tensorflow 1.13.1, CUDA 10, CUDNN 7.4.2, python 3.6 (does not work well with 3.7.. 3.7 has many bugs) For Windows 10 — Ketki Shroff, Mar 16 '19 at 21:12
This is probably all you need for the CUDNN/CUDA version compatibility information for prebuilt packages: https://www.tensorflow.org/install/source#gpu — Jongwook Choi, Jul 21 '20 at 04:12
For the last current releases, this could be useful: https://www.tensorflow.org/install/gpu - possibly use e.g. the wayback machine to go back in time? — Cadoiz, Jul 25 '20 at 14:35

whiletrue · Accepted Answer · 2020-12-05T09:36:29.697

388

TL;DR) See this table: https://www.tensorflow.org/install/source#gpu

Generally:

Check the CUDA version:

cat /usr/local/cuda/version.txt

and cuDNN version:

grep CUDNN_MAJOR -A 2 /usr/local/cuda/include/cudnn.h

and install a combination as given below in the images or here.

The following images and the link provide an overview of the officially supported/tested combinations of CUDA and TensorFlow on Linux, macOS and Windows:

Minor configurations:

Since the given specifications below in some cases might be too broad, here is one specific configuration that works:

tensorflow-gpu==1.12.0
cuda==9.0
cuDNN==7.1.4

The corresponding cudnn can be downloaded here.

Tested build configurations

Please refer to https://www.tensorflow.org/install/source#gpu for a up-to-date compatibility chart (for official TF wheels).

(figures updated May 20, 2020)

Linux GPU

Linux CPU

macOS GPU

macOS CPU

Windows GPU

Windows CPU

Updated as of Dec 5 2020: For the updated information please refer Link for Linux and Link for Windows.

edited Dec 05 '20 at 09:36

answered May 31 '18 at 10:48

whiletrue

10,500
6
27
47

1

I did notice though that TensorFlow versions < 1.0 have been excluded from the overview. Does somebody have an idea where to find the same list for older versions? – whiletrue May 31 '18 at 10:49
3

Looks like they don't specify minor versions for cuda and cudnn, – mrgloom Nov 30 '18 at 00:58
We could maintain a list of minor configurations here. Feel free to edit the post and add working configurations that you have tested. – whiletrue Jan 21 '19 at 20:30
UPDATE: Now tensorflow-gpu==1.12.0 requires cuDNN version 7.2.1 or higher on Windows. – pafi Feb 12 '19 at 13:53
1

UPDATE: tested TF-GPU 1.12, Windows 10, CUDA 9.0, CuDNN 7.3.1, Python 3.6.6 – mjaniec Feb 14 '19 at 07:03
It seems the link on "The corresponding cudnn can be downloaded here." is outdated. – jottbe Aug 08 '19 at 08:50
4

Don't update the figures, link to the documentation. The link changes less often than the tables – Trylks Nov 07 '19 at 14:51
1

@Fábio: Updated your answer with the Latest Links as per your request. – Jan 14 '20 at 10:48
I believe provided images with libs combinations are simply a list of *one* possible combination for each given TF version, which was compiled for downloading by the repo holders. Others are possible and compatible, perhaps. – ivan866 Aug 11 '20 at 13:42
anyone interested in updating the figures? :-) – whiletrue Sep 01 '22 at 08:44
[The table](https://www.tensorflow.org/install/source#gpu) covers only up to TF 2.6.0/CUDA 11.2, however, Tensorflow is already in 2.10. Any explanation to this gap? – Rodrigo Laguna Oct 25 '22 at 19:42

Atul Balaji · Answer 2 · 2019-12-15T08:40:16.360

The compatibility table given in the tensorflow site does not contain specific minor versions for cuda and cuDNN. However, if the specific versions are not met, there will be an error when you try to use tensorflow.

For tensorflow-gpu==1.12.0 and cuda==9.0, the compatible cuDNN version is 7.1.4, which can be downloaded from here after registration.

You can check your cuda version using
nvcc --version

cuDNN version using
cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2

tensorflow-gpu version using
pip freeze | grep tensorflow-gpu

UPDATE: Since tensorflow 2.0, has been released, I will share the compatible cuda and cuDNN versions for it as well (for Ubuntu 18.04).

tensorflow-gpu = 2.0.0
cuda = 10.0
cuDNN = 7.6.0

Is it possible to train a TF1.4 model on a CUDA10 gpu card? – ivan866 Aug 11 '20 at 13:45 — ivan866, Aug 11 '20 at 13:45

score 5 · Answer 3 · answered Mar 28 '20 at 15:15

if you are coding in jupyter notebook, and want to check which cuda version tf is using, run the follow command directly into jupyter cell:

!conda list cudatoolkit

!conda list cudnn

and to check if the gpu is visible to tf:

tf.test.is_gpu_available(
    cuda_only=False, min_cuda_compute_capability=None
)

score 4 · Answer 4 · edited Nov 10 '22 at 06:46

4

Thanks for the first answer.

Something about backward compatibility.

I can successfully install tensorflow-2.4.0 with cuda-11.1 and cudnn 8.0.5.

Source: https://www.tensorflow.org/install/source#gpu

edited Nov 10 '22 at 06:46

Intrastellar Explorer

3,005
9
52
119

answered Apr 03 '22 at 02:52

one

2,205
1
15
37

Hello, can you tell me where did you find this chart please? When I click on the link provided in the messages above the chart stops at Tensorflow 2.6... – Valentin Goldité Apr 22 '22 at 12:31
https://www.tensorflow.org/install/source#gpu – one Apr 22 '22 at 13:11

score 3 · Answer 5 · edited Mar 18 '19 at 17:17

3

You can use this configuration for cuda 10.0 (10.1 does not work as of 3/18), this runs for me:

tensorflow>=1.12.0
tensorflow_gpu>=1.4

Install version tensorflow gpu:

pip install tensorflow-gpu==1.4.0

edited Mar 18 '19 at 17:17

Evan Mata

500
1
6
19

answered Jan 21 '19 at 20:26

ED-DOUGHMI younes

67
3

So, that effectively means that newer gpu cards do NOT support older software stacks, meaning the backwards compatibility is broken indeed? #NVIDIA – ivan866 Aug 11 '20 at 13:51

score 1 · Answer 6 · answered Sep 10 '19 at 15:44

I had installed CUDA 10.1 and CUDNN 7.6 by mistake. You can use following configurations (This worked for me - as of 9/10). :

Tensorflow-gpu == 1.14.0
CUDA 10.1
CUDNN 7.6
Ubuntu 18.04

But I had to create symlinks for it to work as tensorflow originally works with CUDA 10.

sudo ln -s /opt/cuda/targets/x86_64-linux/lib/libcublas.so /opt/cuda/targets/x86_64-linux/lib/libcublas.so.10.0
sudo cp /usr/lib/x86_64-linux-gnu/libcublas.so.10 /usr/local/cuda-10.1/lib64/
sudo ln -s /usr/local/cuda-10.1/lib64/libcublas.so.10 /usr/local/cuda-10.1/lib64/libcublas.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcusolver.so.10 /usr/local/cuda/lib64/libcusolver.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcurand.so.10 /usr/local/cuda/lib64/libcurand.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcufft.so.10 /usr/local/cuda/lib64/libcufft.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda/lib64/libcudart.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcusparse.so.10 /usr/local/cuda/lib64/libcusparse.so.10.0

And add the following to my ~/.bashrc -

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-10.1/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/targets/x86_64-linux/lib/

I believe utilising `nvidia-docker` software layer can deal with the problem of incompatible cuda-tf combinations. Describing how you found out which libs need to be rerouted with symlinks will also be good. — ivan866, Aug 11 '20 at 13:47

score -2 · Answer 7 · edited Jun 25 '20 at 14:12

I had a similar problem after upgrading to TF 2.0. The CUDA version that TF was reporting did not match what Ubuntu 18.04 thought I had installed. It said I was using CUDA 7.5.0, but apt thought I had the right version installed.

What I eventually had to do was grep recursively in /usr/local for CUDNN_MAJOR, and I found that /usr/local/cuda-10.0/targets/x86_64-linux/include/cudnn.h did indeed specify the version as 7.5.0.
/usr/local/cuda-10.1 got it right, and /usr/local/cuda pointed to /usr/local/cuda-10.1, so it was (and remains) a mystery to me why TF was looking at /usr/local/cuda-10.0.

Anyway, I just moved /usr/local/cuda-10.0 to /usr/local/old-cuda-10.0 so TF couldn't find it any more and everything then worked like a charm.

It was all very frustrating, and I still feel like I just did a random hack. But it worked :) and perhaps this will help someone with a similar issue.

10.0 is CUDA SDK version; 7.5 is Compute Capability of the GPU itself — ivan866, Aug 11 '20 at 13:36