2

I'm experimenting with a home cluster, but I would like to debug it in IntelliJ.

The SnappyData documentation says that, in Local Mode, I can create a SnappySession inside my driving program, which based on my Spark experience says I can run it locally while connecting to a Spark cluster. That's great for debugging.

val spark: SparkSession = SparkSession
         .builder
         .appName("SparkApp")
         .master("local[*]")
         .getOrCreate
val snappy = new SnappySession(spark.sparkContext)

But as I start working with more data, I'd still like to be able to debug, only while testing out my cluster's performance as well. For that, I'd like to be able to do the same thing while connecting to a cluster in Embedded Mode. Given that the Lead hosts a Spark driver, or maybe what I'm looking for is on the Server, is there a system I can connect to with the same code, when I'm running an Embedded Mode cluster? How do I find the address and port of that system? (I don't get much command-line feedback from Snappy-Start-All.sh.)

Joseph Pride
  • 165
  • 11
  • Another reason to use the full cluster, even for small jobs, is to "practice how I play" with regards to managing my JARs and dependencies in the cluster. – Joseph Pride Oct 25 '17 at 16:05

0 Answers0