3

I am trying to obtain a DSEGraphFrame of my DSE graphs in either java or scala. I am using the blog documentation, as follows

//load a graph in Java
DseGraphFrame graph = DseGraphFrameBuilder.dseGraph("test", spark); 


// load a graph in scala
val graph = spark.dseGraph("test_graph")

They both require a spark session (in scala is implicit). My question is how can you create the SparkSession spark? I tried already creating it by myself but the builder could not parse the master:

val spark  = SparkSession
.builder
.master("dse://<ip_address>")
.appName("DseGraphFrames")
.getOrCreate()
Mr_Destruction
  • 63
  • 5

1 Answers1

1

Only applications launched with dse spark-submit can understand dse:// master addresses. So make sure your launch is run by dse spark-submit

DSE Doc Reference

RussS
  • 16,476
  • 1
  • 34
  • 62
  • I get `WARN 2017-08-11 16:25:58 org.apache.spark.deploy.SparkNodeConfiguration: Failed to fetch dynamic configuration from DSE` and the job is never executed, it never starts. – Mr_Destruction Aug 11 '17 at 16:30
  • That means the node is either not analytics enabled or there is some other network or authentication problem – RussS Aug 11 '17 at 16:31
  • Analytics is enabled and I authentication should be ok since I can run Gremlin OLAP queries – Mr_Destruction Aug 11 '17 at 19:40
  • that is not necessarily true ... You can always test quickly with the Spark REPL. Try dse spark, it will auto create the spark session object for you. But if you have auth that you aren't passing through or setting up it will fail to start. – RussS Aug 11 '17 at 19:49