1

I recently started using mac and there I have installed pyspark. Now whenever I try to create a SparkSession .

spark = (
    SparkSession.builder
        .getOrCreate()
)

It throws me an error if I execute/debug it from pycharm although it works fine if i run the python program via terminal eg: python main.py.

The error it throws is:

The operation couldn’t be completed. Unable to locate a Java Runtime. Please visit http://www.java.com for information on installing Java.

/usr/local/lib/python3.9/site-packages/pyspark/bin/spark-class: line 96: CMD: bad array subscript head: illegal line count -- -1

RuntimeError: Java gateway process exited before sending its port number

Can someone help me on this ?

vaibhav
  • 45
  • 6
  • it might be the case where pycharm doesn't know the path to the java executable. – samkart Jul 18 '22 at 13:03
  • I tried adding the java path using this link https://www.jetbrains.com/help/pycharm/switching-boot-jdk.html , but it didn't work. – vaibhav Jul 20 '22 at 09:48
  • You probably don't have Java installed on your system. If you want to install with brew, here's how to do it: https://stackoverflow.com/questions/65601196/how-to-brew-install-java/70786302#70786302 – Cornelius Roemer Aug 12 '22 at 19:55
  • Java is installed – vaibhav Aug 14 '22 at 04:18

0 Answers0