1

I have installed Cloudera VM and hence it has PySpark with MLLIB library, but the ML library MLLIB is too old, I just wanted to upgrade it with latest version of MLLIB, Already updated the python from 2.6 to 2.7, but unable to find any documentation to upgrade MLLIB.

Thanks

Krishna

zero323
  • 322,348
  • 103
  • 959
  • 935
krishna Prasad
  • 3,541
  • 1
  • 34
  • 44

1 Answers1

3

Different Spark components are tightly connected and in general are not compatible between versions so you should simply update complete Spark distribution. Spark Downloads page provides a number of binaries pre-built with different Hadoop versions.

zero323
  • 322,348
  • 103
  • 959
  • 935
  • Thanks for your reply, if possible can you please provide how should I update completed Spark distribution. – krishna Prasad Mar 19 '16 at 16:22
  • 1
    Go to the linked download page, choose desired binaries, download, unpack. Optionally copy existing configuration. – zero323 Mar 19 '16 at 18:13
  • 1
    I have put some instructions on how to "upgrade" Spark here http://stackoverflow.com/questions/33887227/how-to-upgrade-spark-to-newer-version/33914992#33914992 – desertnaut Apr 20 '16 at 13:15