scalding NoClassDefFoundError

Question

after installing java, scala scalding on OSX i have the following error when i try to run the WordCountJob file

$ scripts/scald.rb --local  WordCountJob.scala --input someInputfile.txt --output ./someOutputFile.tsv

compiling WordCountJob.scala scalac -classpath /Users/me/.sbt/boot/scala-2.9.3/lib/scala-library.jar:/Drive/me/Coding/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.9.1.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/hadoop-core-1.1.2.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-codec-1.8.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-configuration-1.9.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/jackson-asl-0.9.5.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/jackson-mapper-asl-1.9.13.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-lang-2.6.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/slf4j-log4j12-1.6.6.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/log4j-1.2.15.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-httpclient-3.1.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-cli-1.2.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/commons-logging-1.1.1.jar:/var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/maven/zookeeper-3.3.4.jar -d /var/folders/5q/7bth9vvd2tv8q57f737f6jxm0000gn/T/script-build WordCountJob.scala Exception in thread "main" java.lang.NoClassDefFoundError: scala/tools/nsc/Main Caused by: java.lang.ClassNotFoundException: scala.tools.nsc.Main at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

does anybody know how to solve this ?

score 0 · Answer 1 · answered May 02 '14 at 16:07

0

It cannot find Scala compiler. You need to set SCALA_HOME and add ./bin directory to your PATH:

 export SCALA_HOME=<where you installed scala>
 export PATH=$PATH:$SCALA_HOME/bin

answered May 02 '14 at 16:07

Sasha O

3,710
2
35
45

i've put my stb assembly and scald.rb errors in the comment above – john Stilkowicz May 02 '14 at 18:08
You can also try to go a different route: use SBT only. Look at https://github.com/vitalyg/Strata2014ScaldingTutorial . Possibly you need to update scalding version number in project/Dependencies.scala – Sasha O May 02 '14 at 20:42
also try ./scald.rb --clean when something like this goes on. Sometimes the cached classes are corrupt. – Oscar Boykin Aug 01 '14 at 20:35

score 0 · Answer 2 · edited May 23 '17 at 11:57

on some forums i saw that in JRE 7 this error is not present so i've updated to jre 7

$ java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)

the only way to make it active

export JAVA_HOME="/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home"

according to Installing Java on OS X 10.9 (Mavericks)

$ echo $JAVA_HOME
/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home

I added the following lines to ~/.profile

export JAVA_HOME=$(/usr/libexec/java_home)
export SCALA_HOME="/usr/local/Cellar/scala/2.9.3"
export PATH=$PATH:$SCALA_HOME/bin

In /scalding I want to run -> sbt assembly and having the following error

[info] Passed: Total 51, Failed 0, Errors 0, Passed 51
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Strategy 'discard' was applied to a file
[info] Assembly up to date: /Drive/me/Coding/scalding/scalding-date/target/scala-2.9.3/scalding-date-assembly-0.9.1.jar
[error] (maple/compile:compile) javac returned nonzero exit code
[error] Total time: 29 s, completed May 2, 2014 8:27:30 PM

When trying to run WordCountJob I have the following error

$ scripts/scald.rb --local WordCountJob.scala --input someInputfile.txt --output ./someOutputFile.tsv
/Drive/me/Coding/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.9.1.jar is missing, you probably need to run sbt assembly

It look like a mission imposible to have scalding running in local mode.

scalding NoClassDefFoundError

2 Answers2