Stack Overflow
Stack Overflow

Questions tagged [multiple-gpu]

See the tag entry for "multi-gpu".

43 questions
7
votes
1 answer

Is it possible to run a cuda kernel on multiple gpus

This is a fairly simple question but googling doesn't seem to have the answer, so. What I want to know is if I have two gpu cards (identical) capable of running cuda, can my kernel span these cards? Or is it bound to one card or the other? I.e. is…
user257111
6
votes
2 answers

Concurrency in CUDA multi-GPU executions

I'm running a cuda kernel function on a multiple GPUs system, with 4 GPUs. I've expected them to be launched concurrently, but they are not. I measure the starting time of each kernel, and the second kernel starts after the first one finishes its…
user1555209
  • 73
  • 2
  • 7
5
votes
2 answers

How to continue training after loading model on multiple GPUs in Tensorflow 2.0 with Keras API?

I trained a text classification model consisting RNN in Tensorflow 2.0 with Keras API. I trained this model on multiple GPUs(2) using tf.distribute.MirroredStrategy() from here. I saved the checkpoint of the model using…
Rishabh Sahrawat
  • 2,437
  • 1
  • 15
  • 32
4
votes
1 answer

Any new ideas on using openCL with multiple GPUs?

My question is : Has there been any new advancement (or perhaps a tool/library developed) for using openCL with multiple GPUs? I understand that if someone wants to write a code in openCL with the goal of using multiple GPUs, then he can, but I…
Kostis
  • 447
  • 4
  • 13
4
votes
2 answers

PyTorch custom forward function does not work with DataParallel

Edit: I have tried PyTorch 1.6.0 and 1.7.1, both give me the same error. I have a model that allows users to easily switch between different architectures A and B. The forward functions for both architectures are different too, so I have the…
Raven Cheuk
  • 2,903
  • 4
  • 27
  • 54
4
votes
2 answers

Scheduling GPU resources using the Sun Grid Engine (SGE)

We have a cluster of machines, each with 4 GPUs. Each job should be able to ask for 1-4 GPUs. Here's the catch: I would like the SGE to tell each job which GPU(s) it should take. Unlike the CPU, a GPU works best if only one process accesses it at…
Daniel Blezek
  • 4,539
  • 1
  • 19
  • 20
3
votes
1 answer

Is there something like Hadoop, but based on GPU?

Is there something like Hadoop, but based on GPU? I would like to do some research on distributed computing. Thank you for your help! Yik,
Yik
  • 515
  • 1
  • 5
  • 16
3
votes
0 answers

How does the Windows 10 render windows under multi-display, multi-GPU environment?

I am working with four displays and both external and internal GPU enabled (IGFX enabled). Three of them are connected to external GPU and the other is connected to motherboard. In graphics control panel, I see that each display is allocated to…
Eli
  • 53
  • 3
3
votes
1 answer

How to understand "All threads in a warp execute the same instruction at the same time." in GPU?

I am reading Professional CUDA C Programming, and in GPU Architecture Overview section: CUDA employs a Single Instruction Multiple Thread (SIMT) architecture to manage and execute threads in groups of 32 called warps. All threads in a warp execute…
Nan Xiao
  • 16,671
  • 18
  • 103
  • 164
3
votes
3 answers

Running OpenCL kernel on multiple GPUs?

Right now I programmed made several algorithms running in parallel on one GPU, but all of them have the same problem, when I try to execute them on several GPUs (for example, 3). The problem is that the code, executed on one GPU executes exactly the…
Vladimir
  • 457
  • 6
  • 17
2
votes
0 answers

Getting ProcessExitedException. How to spawn multiple processes on databricks notebook using torch.multiprocessing?

I am trying out distributed training in pytorch using "DistributedDataParallel" strategy on databrick notebooks (or any notebooks environment). But I am stuck with multi-processing on a databricks notebook environment. Problem: I want to spwan…
sarjit07
  • 7,511
  • 1
  • 17
  • 15
2
votes
0 answers

LSTM model Tensorflow 2.1.0 tf.distribute.MirroredStrategy() slow on AWS instance g3.4large

I am running an LSTM model on AWS instance g3.8xlarge which has 2 GPUs, and using tf.distribute.MirroredStrategy() so that I can use the 2 GPUs. However, the training time is actually slower than without using this. Does anyone know how to solve…
sunday
  • 31
  • 1
2
votes
1 answer

How to use multiple GPU for a DCGAN using Tensorflow 2.0 - RuntimeError: Replica-local variables may only be assigned in a replica context

I would like to develop a DCGAN with resolution of 256x256. To do so I need to use multiple GPU since only one it is not enough and it will probably take too much time. I followed the procedure explained in the documentation at this…
nbl7
  • 511
  • 4
  • 14
2
votes
1 answer

How to speed up the training of an RNN model with multiple GPUs in TensorFlow?

For example, the RNN is a dynamic 3-layer bidirectional LSTM with the hidden vector size of 200 (tf.nn.bidirectional_dynamic_rnn) and I have 4 GPUs to train the model. I saw a post using data parallelism on subsets of samples in a batch but that…
Maosi Chen
  • 1,492
  • 2
  • 14
  • 33
2
votes
1 answer

Can I use two NVIDIA gpu cards in a system without SLI support for CUDA computation?

My current system for CUDA applications has one old NVIDIA card, 8800 GTX. I am thinking of adding one more card to it without updating the motherboard. Is it true that as long as I have two PCI-E slots, the two will work? Or I have to purchase a…
fflower
  • 161
  • 1
  • 2
  • 7
1
2 3