1

I uploaded a jar library on my cluster in Databricks following this tutorial, however I have been unable to import the library or use the methods of the library from the Databricks notebook. I have been unable to find forums or documentation that address this topic, so I'm unsure if it's even possible at this point.

I am able to run the jar file as a job in Databricks, I just haven't been able to import the jar library into the Notebook to run it from there.

I also tried running the jar file using the %sh magic command but received the following JNI error:

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
D.I.
  • 31
  • 6
  • 1
    Update: running the jar using the %sh command didn't work as java code cannot be run in the Databricks notebook, hence the error. See: https://azurelib.com/how-to-run-java-code-on-azure-databricks/ – D.I. Dec 30 '21 at 19:09
  • you can refer for complete solution here . https://stackoverflow.com/questions/60543850/how-to-install-a-library-on-a-databricks-cluster-using-some-command-in-the-noteb – Karthikeyan Rasipalay Durairaj Dec 30 '21 at 19:36

1 Answers1

0

This error generally caused because of Scala version. I would recommend upgrading Scala version and then try to import custom library.

Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0

Refer - Apache Spark Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class

Abhishek K
  • 3,047
  • 1
  • 6
  • 19