1

I need a powerful system to work with neural networks. I decided to use CUDA-based cuDNN library. Now I wonder whether it supports SLI technology. I read original tutorials about cuDNN and found no mention of SLI. I even haven't found a solid answer whether CUDA itself supports SLI technology.

To be more short:

1) Does cuDNN library support SLI?
 2) If the answer is negative for the question above, are there any ways not to be restricted with only one graphic card?

Max
  • 101
  • 1
  • 12
  • CUDA doesn't support SLI. cuDNN has baked in multiple GPU support – talonmies Dec 07 '15 at 16:43
  • 1
    1 - No. 2 - No. 3 - Yes, for the general CUDA case, CUDA provides the ability to use multiple GPUs, for an example refer to the [simpleMultGPU](http://docs.nvidia.com/cuda/cuda-samples/index.html#simple-multi-gpu) CUDA sample code. For the cuDNN case, typical cuDNN multi-GPU usage would involve multiple host threads using cuDNN; it is thread-safe and permits this. The topic of multi-GPU in the general neural networks case is a complex one and will depend on whether you need model parallelism or just data parallelism. – Robert Crovella Dec 07 '15 at 16:43
  • @talonmies, so you mean that cuDNN supports multiple GPUs (based on SLI) and handles them automatically. Did I get you right? P.S. Please, unmark this question as a duplicate. Your referral is about CUDA and my question is about cuDNN for the most part. – Max Dec 07 '15 at 16:48
  • 1
    @MaxTsylko it's probably better if you ask a focused, more refined question. Item #1 in your question has nothing to do with cuDNN, and the answer for item #2 is implied by the answer for item #1, since the answer is negative for item 1. **Niether CUDA nor cuDNN use SLI.** Multi-GPU and SLI are not synonymous. SLI is one possible approach for data sharing between GPUs; CUDA (and cuDNN) do not use it. They employ other methods for data sharing between GPUs, and to support use of multiple GPUs in general. – Robert Crovella Dec 07 '15 at 16:56
  • @Robert Crovella, you are right, I took into consideration what you said and edited my question. I hope that it looks a bit more logical now. I understand the difference between multi-GPU and CUDA. If I got you right, you suggest to separate the data explicitly and then use the different GPUs? – Max Dec 07 '15 at 17:11
  • cuDNN doesn't support SLI. Hasn't that question been answered already? Note the duplicate text: "If those answers do not fully address your question, please ask a **new** question." – Robert Crovella Dec 07 '15 at 17:16
  • @Robert Crovella, I completely understood your answer about SLI. As for duplicating, I think there is no need in creating one more question. I edited this question to avoid duplication and it's allowed by the rules. "If those answers do not fully address your question, please edit this question to explain how it is different **or** ask a new question." – Max Dec 07 '15 at 17:25

0 Answers0