8

I am currently learning programming with GPU to improve the performance of machine learning algorithms. Initially I try to learn programming cuda with pure c, then I found pycuda which to me a wrapper of cuda library, and then I found theano and pylearn2 and got a little confused:

I understand them in this way:

  1. pycuda: python wrapper for cuda library
  2. theano: similar to numpy but transparent to GPU and CPU
  3. pylearn2: deep learning package which build on theano and implemented several machine learning/deep learning model

Since I am new to GPU programming, should I start learning from C/C++ implementation or starting from pycuda is enough, even starting from theano? E.g. I would like to implement randomForest model after learning the GPU programming.Thanks.

user1754197
  • 127
  • 1
  • 9

1 Answers1

9

Your understand is almost right. I would just add some remarks about Theano. It's much more than a Numpy which can run on the GPU. Theano is indeed is math expression compiler, which translates symbolic math expressions in highly optimized C/CUDA code, targeted for both CPU and GPU. The code it generates is often much more efficient than the one most programmers would write. Theano also can make symbolic differentiation (very useful for gradient based optimization) and has also a feature to achieve better numerical stability (which probably is something useful, though I don't know for real to what extent). It's very likely Theano will be enough to implement what you need. If you still decide to learn CUDA or PyCUDA, choose the one based no the language you will use, C++ or Python.

Saul Berardo
  • 2,610
  • 3
  • 20
  • 24
  • Very clear, Thanks! One more question but probably not suitable to ask it here: is there any similar package as theano in R language? – user1754197 Sep 18 '14 at 13:36
  • Not quite. There are a few packages which implement basic linear algebra operations and some higher level functions (as lm), but under the hood they work very differently from Theano, i.e. they are just regular packages, they are not expression compilers. – Saul Berardo Sep 18 '14 at 16:38