When I initiate pyspark on pycharm / jupyter notebook, I just keep getting this error:
RuntimeError: Java gateway process exited before sending its port number
I did the following:
- pip installed pyspark,
- installed java 8, added environment variables,
- tested java on cmd, input
java
,java -version
,javac
, all okay - in my code, I added
os.environ['JAVA_HOME'] = 'D:\java'
, not working - added
os.environ['PYSPARK_SUBMIT_ARGS'] = "--master local"
not working
Pretty sure java 8 was set properly, however just kept getting this error.
please help