9

I'm using Python to access Databricks through databricks-connect. Behind the wall, this uses spark which is indeed java based so in order to use this, I need java. The JDK has been downloaded (version 14), set as JAVA_HOME env but when I run the code, I get the following error:

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module @bccb269

This is the code where it crashes

from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()

I googled but couldn't find this error, I don't think it's the first time occurring to me. Any ideas what this error means and how can I fix it?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
anthino12
  • 770
  • 1
  • 6
  • 29
  • 1
    It means that library is trying to hack a Java core library, which is not allowed starting in Java 9. The easiest solution is to stick with Java 8. – VGR Nov 29 '21 at 14:13

2 Answers2

9

Databricks Runtimes and databricks-connect won't work with Java 14. Only DBR 10.x have an experimental support for Java 11, but I doubt that it's supported for databricks-connect.

You need to install Java 8 to use databricks-connect.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
0

I had to

  • Install Java 8, eg
brew install --cask adoptopenjdk/openjdk/adoptopenjdk8
  • add $JAVA_HOME to my ~/.zshrc
export JAVA_HOME='/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home'
export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)
  • remove any other jdk installs
ls /Library/Java/JavaVirtualMachines
sudo rm -rf NONJDK8.jdk

The 3rd bit is important! It did not work until I removed other-versioned jdks.

ehacinom
  • 8,070
  • 7
  • 43
  • 65