9

I am trying to connect to Neo4j from Spark using neo4j-spark-connector. I am facing an authentication issue when I try to connect to the Neo4j org.neo4j.driver.v1.exceptions.AuthenticationException: Unsupported authentication token, scheme='none' only allowed when auth is disabled: { scheme='none' }

I have checked and the credentials I am passing are correct. Not sure why is it failing.

import org.neo4j.spark._
import org.apache.spark._
import org.graphframes._
import org.apache.spark.sql.SparkSession
import org.neo4j.driver.v1.GraphDatabase
import org.neo4j.driver.v1.AuthTokens

val config = new SparkConf()

config.set(Neo4jConfig.prefix + "url", "bolt://localhost")
config.set(Neo4jConfig.prefix + "user", "neo4j")
config.set(Neo4jConfig.prefix + "password", "root")

val sparkSession :SparkSession = SparkSession.builder.config(config).getOrCreate()

val neo = Neo4j(sparkSession.sparkContext)

val graphFrame = neo.pattern(("Person","id"),("KNOWS","null"), ("Employee","id")).partitions(3).rows(1000).loadGraphFrame


println("**********Graphframe Vertices Count************")
graphFrame.vertices.count

println("**********Graphframe Edges Count************")
graphFrame.edges.count


val pageRankFrame = graphFrame.pageRank.maxIter(5).run()
val ranked = pageRankFrame.vertices
ranked.printSchema()

val top3 = ranked.orderBy(ranked.col("pagerank").desc).take(3)

Can someone please have a look and let me know the reason for the same?

Mitaksh Gupta
  • 1,029
  • 6
  • 24
  • 50
  • What's `Neo4jConfig.prefix`? – František Hartman Oct 14 '19 at 14:46
  • `Neo4jConfig.prefix` is the prefix for setting the neo4j specific properties in spark. Its value comes from the neo4j-spark connector's code, and is "spark.neo4j.bolt." Reference - https://github.com/neo4j-contrib/neo4j-spark-connector/blob/master/src/main/scala/org/neo4j/spark/Neo4jConfig.scala – Mitaksh Gupta Oct 15 '19 at 05:24
  • I believe you forgot the port number: `bolt://localhost:7687` – RudyVerboven Oct 17 '19 at 09:07
  • @RudyVerboven - It doesn't work with the port number as well. I have already tried that. – Mitaksh Gupta Oct 17 '19 at 09:54
  • How are you running your spark application ? Have you tried setting your credentials in the conf/spark-defaults.conf ? – RudyVerboven Oct 17 '19 at 15:33
  • Is it possible that you already have a `SparkSession` when you call `SparkSession.builder.config(config).getOrCreate()`? In that case your added configs might not have any effect. – mazaneicha Oct 23 '19 at 22:57
  • @mazaneicha - It maybe possible. How can I check that? My understanding of the getOrCreate() method was that it gets the existing SparkSession and sets the property there. But if that's not the case then how can I correct that? – Mitaksh Gupta Oct 24 '19 at 05:21
  • @MitakshGupta see https://stackoverflow.com/questions/41886346/spark-2-1-0-session-config-settings-pyspark, and translate to scala (or google for scala specifically :)) – mazaneicha Oct 24 '19 at 12:40
  • @mazaneicha - This getAll() method doesn't work for me. I was however able to print all the configurations using `sc.getConf.toDebugString` but it didn't print the Neo4j specific properties in it. Researching further I found out that it only prints Spark's default properties, and not the runtime properties that you set. – Mitaksh Gupta Oct 25 '19 at 05:02
  • @RudyVerboven - Looks like I miised responding to your comment. Sorry for that. I tried setting the credentials in default config as well, and it didn't work for me. – Mitaksh Gupta Oct 25 '19 at 05:22
  • @MitakshGupta The point of that post is that you can't add settings to existing session, rather you should create a config that you need > stop current session > use your config to start new session. getAll allows you to bootstrap current config and simply append additional settings. And it surely does work! `scala> val currentconf = sc.getConf.getAll` > `currentconf: Array[(String, String)] = Array((spark.master,yarn), (spark.app.name,SparkShell),...)` – mazaneicha Oct 25 '19 at 13:13

2 Answers2

1

It might be a configuration issue with your neo4j.conf file. Is this line commented out:

dbms.security.auth_enabled=false

Community
  • 1
  • 1
Dave Fauth
  • 828
  • 5
  • 3
  • This is not commented and its value is set to true. When I set the value as false, it will mean that I have disabled the authentication and that's when my script works fine too. What shall be the value set for this variable? – Mitaksh Gupta Oct 17 '19 at 07:15
1

I had a similar problem, creating the following spring beans fixed the issue.

    @Bean
public org.neo4j.ogm.config.Configuration getConfiguration() {
    return new org.neo4j.ogm.config.Configuration.Builder()
            .credentials("neo4j", "secret")
            .uri("bolt://localhost:7687").build();
}

@Bean
public SessionFactory sessionFactory(org.neo4j.ogm.config.Configuration configuration) {
    return new SessionFactory(configuration,
            "<your base package>");
}
lizom
  • 41
  • 7