23

Using Word2vec and Doc2vec methods provided by Gensim, they have a distributed version which uses BLAS, ATLAS, etc to speedup (details here). However, is it supporting GPU mode? Is it possible to get GPU working if using Gensim?

Irene Li
  • 341
  • 1
  • 3
  • 6

1 Answers1

21

Thank you for your question. Using GPU is on the Gensim roadmap. Will appreciate any input that you have about it.

There is a version of word2vec running on keras by @niitsuma called word2veckeras. The code that runs on latest Keras version is in this fork and branch https://github.com/SimonPavlik/word2vec-keras-in-gensim/tree/keras106

@SimonPavlik has run performance test on this code. He found that a single gpu is slower than multiple CPUs for word2vec.

Regards Lev

Lev Konst
  • 241
  • 1
  • 3
  • Hi Lev, Thanks for the link! It is good to know there is a Keras version (although it is still using Gensim I suppose). Maybe I could search for other word2vec/doc2vec versions as well. Anyway, looking forward to the GPU Gensim :D Thanks Irene – Irene Li Sep 30 '16 at 12:59
  • 13
    Hi @Lev What's the current status of this? What kind of performance improvements is this supposed to bring? – lucid_dreamer Jun 17 '17 at 01:58
  • [Here](https://github.com/RaRe-Technologies/gensim/issues/449) is the issue: Word2vec to use GPU – Ohad Bruker Jul 31 '19 at 09:58
  • We need to consider the relevance of reproducible results. Using FastText it is possible using only one thread. I believe that Keras has the same problem. But using Pytorch is possible to reproduce the exact same training process setting torch.manual_seed(seed). This command will affect all devices (CPU or GPU), but some commands can insert random behavior https://pytorch.org/docs/stable/notes/randomness.html. – Mello Feb 15 '21 at 17:09
  • 1
    Note: Gensim has no plans to implement GPU support, unfortunately. See [here](https://github.com/RaRe-Technologies/gensim/issues/449#issuecomment-618632186). – Dan Nissenbaum Oct 16 '21 at 23:41