3

I'm working with word embeddings using PySpark, in particular I'm working with Word2Vec. Now, I'd like to try Glove instead of Word2Vec but, apparently, it doesn't exist an implementation of Glove with PySpark, but only with Scala language.

Is there any way to use Glove with PySpark? Can I use the Scala implementation into PySpark, or am I obliged to implement from scratch the algorithm?

Thanks

Davide
  • 73
  • 5
  • Did you found a way? – Maviles Sep 08 '20 at 12:24
  • there is a way to invoke this in PySpark.. checkout [ https://github.com/dmarcous/spark-glove ], you can start spark with pyspark `--driver-class-path` argument and pass the glove.jar dependency (from dmarcous's git repo) and invoke it in pyspark using `output_variable = sc._jvm.com.glove.GloVeRunner(# pass arguments here)` – Anantha Sharma Sep 08 '20 at 21:43

0 Answers0