4

I am trying to visualize decision tree structure in pyspark. But all the tools are for data. I could not find any for visualizing tree structure. Or is there a way I can visualize using the rules from toDebugString?

Timbus Calin
  • 13,809
  • 5
  • 41
  • 59
kabraxis
  • 150
  • 2
  • 8
  • 4
    Could there be a way to port pyspark decision tree to scikit ad then use sklearn to generate graphviz dot file? – Cediddi Mar 21 '16 at 19:56

2 Answers2

5

I have tried to do the following in order to create a visualization :

  1. Parse Spark Decision Tree output to a JSON format.
  2. Use the JSON file as an input to a D3.js visualization.

For more code you can refer to my prototype at GitHub here.

RoyaumeIX
  • 1,947
  • 4
  • 13
  • 37
1

I'm also a bigdata/ml engineer and I had many time the need to visualize the decision trees from Spark.

We've just released dtreeviz 1.1 , which includes support for Spark. You can now visualize the tree structure, leaf information, prediction path and more. Just check this notebook for more examples.

enter image description here enter image description here

Tudor Lapusan
  • 173
  • 7