0

I'm trying to run GraphX on Ipython notebook. Firstly, I launched Spark/Hadoop clusters and then launched ipython notebook using this tutorial (http://blog.insightdatalabs.com/jupyter-on-apache-spark-step-by-step/). But now I have only Python 2 kernel and I need GraphX to work with graph. How can I run it?

Alex Ermolaev
  • 311
  • 2
  • 4
  • 17

1 Answers1

0

GraphX can only be referenced by the Scala API. Saying this you can either use something like Jupyter-Scala so you can use Scala within Jupyter or you can use GraphFrames which supports the Python API. For more information on GraphFrames, please check out Introducing GraphFrames.

Denny Lee
  • 3,154
  • 1
  • 20
  • 33
  • I wanted to install apache toree instead but got the following error: No such file or directory: '/usr/local/spark/python/lib' – Alex Ermolaev Nov 12 '16 at 22:50
  • This may be a case where the default location where Apache Toree is expecting the PYTHON_PATH isn't where you had installed python. Could you check the toree kernel.json and update the "PYTHONPATH": "/usr/local/spark/python:..." to your location. For more information, please check http://technippet.blogspot.com/2016/02/running-apache-spark-in-jupyter-notebook.html – Denny Lee Nov 13 '16 at 02:36
  • I followed this tutorial: launched apache toree kernel, but it doesn't respond ( – Alex Ermolaev Nov 14 '16 at 08:26
  • Could you provide some more context as there isn't enough information here to debug. Saying this - why not use GraphFrames so that way you can use Python? – Denny Lee Nov 15 '16 at 07:15
  • yeah, I think it would be great to use GraphFrames, but how can I integrate it with Ipython? Now in my /usr/local/share/jupyter/kernels I have only apache_toree_scala. – Alex Ermolaev Nov 15 '16 at 09:51
  • You can use GraphFrames with pyspark so it follows that flow. The key thing is to add the call to include the GraphFrames Spark Package. For Apache Toree, you may want to create a new question to get others to help (while I've used it, I haven't used it enough to debug yet). – Denny Lee Nov 15 '16 at 16:53