1

I am a beginner in deep learning/theano/keras.I'm trying to figure out how to use multiple gpus on windows 7. I've had success installing Theano,keras(as described in this post How do I install Keras and Theano in Anaconda Python on Windows?) and using one gpu. I want to use both my gpus

Following are the details of configs and versions

Python - 2.7(Anaconda-4.3.14,Windows-64bit) ,CUDA - 7.5.17 ,Theano - 0.9.0rc3 ,keras - 1.2.2 ,pycuda - 2016.1.2+cuda7518 ,gpu - Geforce GTX 480(2 of them)

Theano configuration is as below .theanorc.txt

[global]
floatX = float32
device = gpu

[nvcc]
flags=-LC:\ProgramData\Anaconda2\libs
compiler_bindir=C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin

[lib]
cnmem=0.8

Currently I'm able to use only one GPU and I am getting memory error as below when I try to fit the model

MemoryError: ('Error allocating 411041792 bytes of device memory (CNMEM_STATUS_OUT_OF_MEMORY).', "you might consider using 'theano.shared(..., borrow=True)'")

Does using 2 gpus solve the problem(if yes, how do I enable the second one?) or is my model too big ?

Thank You

Community
  • 1
  • 1
rteja1113
  • 21
  • 2
  • Reduce `cnmem` value to 0.5 or even to zero first and check if the `CNMEM_STATUS` error goes away. You can do multi-gpu training with Keras+Tensorflow. See [here](https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html) and [here](https://github.com/fchollet/keras/issues/2436). – Autonomous Mar 12 '17 at 00:08
  • Hi @ParagS.Chandakkar, I tried cnmem to zero, and I got this error.MemoryError: ('Error allocating 67108864 bytes of device memory (out of memory).', "you might consider using 'theano.shared(..., borrow=True)'") – rteja1113 Mar 12 '17 at 01:49
  • Hi @ParagS.Chandakkar, I can't install tensorflow on Windows as my GPU's Compute Capability is < 3.0 – rteja1113 Mar 12 '17 at 02:40
  • If you have complex model try to lower down the batchsize it might help with the memory issue – oak Mar 12 '17 at 09:51
  • Hi @oak, My batch size is 5.Should I lower it even further ? any other ideas ? – rteja1113 Mar 12 '17 at 18:37
  • @RaviTeja using `keras` and `batchsize`. `Keras` engine tries to load all the batch size in once. This may cause memory issue. But 5 is not a big batch. How big is your network? – oak Mar 13 '17 at 11:41
  • I actually precomputed the convolution layers.I simply loaded these precomputed conv_layer outputs and used them as input for fully connected layers.The input to the network is 512x14x14 .The network is Maxpool-Flatten-(Dense(4096)-Dropout(0.5)-Batchnorm-)x2 - Softmax(5 units). The stride in maxpool is 2 – rteja1113 Mar 13 '17 at 22:21

0 Answers0