6

This exception is rising at lines.count().

Exception has occurred: py4j.protocol.Py4JError An error occurred while calling o26.isBarrier. Trace: py4j.Py4JException: Method isBarrier([]) does not exist at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318) at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326) at py4j.Gateway.invoke(Gateway.java:274) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79) at from pyspark import SparkContext from pyspark import SparkConf

Code:

    conf = SparkConf()

    conf.setAppName("First App")

    sc = SparkContext('local',conf=conf)
    print("-----------------------------------------------------------------------------")
    lines = sc.textFile("sample.csv")
    print("-----------------------------------------------------------------------------")
    lines.count()
Raduan Santos
  • 1,023
  • 1
  • 21
  • 45
Ankush K
  • 334
  • 3
  • 13
  • This means you're mixing incompatible versions of Spark (JVM side) and PySpark – 10465355 Jan 30 '19 at 10:46
  • i used `pip install pyspark==2.3.2` command to install pyspark. It also installed py4j version 0.10.7 – Ankush K Jan 30 '19 at 10:52
  • At least the Python libraries uses 2.4. See my answer to [Method showString(\[class java.lang.Integer, class java.lang.Integer, class java.lang.Boolean\]) does not exist in PySpark](https://stackoverflow.com/questions/53455489/method-showstringclass-java-lang-integer-class-java-lang-integer-class-java) – 10465355 Jan 30 '19 at 10:55

0 Answers0