42

In shell I typed gradle cleanJar in the Impatient/part1 directory. The output is below. The error is "class file for org.apache.hadoop.mapred.JobConf not found". Why did it fail to compile?

:clean UP-TO-DATE
:compileJava
Download http://conjars.org/repo/cascading/cascading-core/2.0.1/cascading-core-2.0.1.pom
Download http://conjars.org/repo/cascading/cascading-hadoop/2.0.1/cascading-hadoop-2.0.1.pom
Download http://conjars.org/repo/riffle/riffle/0.1-dev/riffle-0.1-dev.pom
Download http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.pom
Download http://repo1.maven.org/maven2/org/slf4j/slf4j-parent/1.6.1/slf4j-parent-1.6.1.pom
Download http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.pom
Download http://conjars.org/repo/thirdparty/jgrapht-jdk1.6/0.8.1/jgrapht-jdk1.6-0.8.1.pom
Download http://repo1.maven.org/maven2/org/codehaus/janino/janino/2.5.16/janino-2.5.16.pom
Download http://conjars.org/repo/cascading/cascading-core/2.0.1/cascading-core-2.0.1.jar
Download http://conjars.org/repo/cascading/cascading-hadoop/2.0.1/cascading-hadoop-2.0.1.jar
Download http://conjars.org/repo/riffle/riffle/0.1-dev/riffle-0.1-dev.jar
Download http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Download http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar
Download http://conjars.org/repo/thirdparty/jgrapht-jdk1.6/0.8.1/jgrapht-jdk1.6-0.8.1.jar
Download http://repo1.maven.org/maven2/org/codehaus/janino/janino/2.5.16/janino-2.5.16.jar
/home/is_admin/lab/cascading/Impatient/part1/src/main/java/impatient/Main.java:50: error: cannot access JobConf
    Tap inTap = new Hfs( new TextDelimited( true, "\t" ), inPath );
                ^
  class file for org.apache.hadoop.mapred.JobConf not found
1 error
:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 22.599 secs
Raedwald
  • 46,613
  • 43
  • 151
  • 237
Treper
  • 3,539
  • 2
  • 26
  • 48
  • 12
    I don't think this should be closed: The problem here is that gradle doesn't play nicely with java 7, and anyone going through the cascading tutorial with 7.x JDK will have this issue. The answer, which I'm happy to add, is to use a Java 6 JDK with the cascading tutorial. – Paul Sanwald Feb 02 '13 at 21:20
  • See also https://groups.google.com/d/topic/cascading-user/XEZapg4uqdI/discussion – hertzsprung Feb 24 '13 at 00:37
  • I believe this is because you don't have hadoop installed, and wordcount/build.gradle has "{ transitive = false }" so it doesn't install it. Remove that, and it'll build. – Gabriel Burt Apr 25 '13 at 22:27
  • I think that class got missed in the jar. please check the jar in your repository. – Learner Jun 25 '13 at 09:34
  • 1
    I think hadoop-conf jar is not present in classpath.Let me know in which hadoop version you are running this program. – user1651008 Jul 15 '13 at 11:11
  • If you are on OSX, you can download Java 6 here - https://developer.apple.com/downloads. Search for Java 6, "Java for OS X 2013-005 Developer Package" works. – bgs Mar 03 '14 at 23:35

5 Answers5

1

try using Java 6 , gradle has some known issues with java 7

user1801279
  • 1,743
  • 5
  • 24
  • 40
1

Add hadoop-core-1.x.y.jar from hadoop distribution to path.

This worked for me in CoPA example. I added to eclipse build path.

Add this to gradle dependencies{} section

compile( group: 'org.apache.hadoop', name: 'hadoop-core', version: '1.2.0' )

Lalit Jha
  • 2,143
  • 1
  • 12
  • 6
0

It seems like class not found, please check libraries in build path entries.

user2826111
  • 152
  • 1
  • 2
  • 13
0

I had the same issue today trying the Cascading for the Impatient tutorial. I have solved adding the following to my build.gradle in the dependenciessection :

compile 'org.apache.hadoop:hadoop-client:2.3.0'

I wanted to use Hadoop 2.3.0, but I guess that you can choose your own version of Hadoop.

PS : In order to use the first hadoop-mapred package, you have to add the following instead :

compile 'org.apache.hadoop:hadoop-mapred:0.22.0'

You can find all the repos you might need for Hadoop here.

eliasah
  • 39,588
  • 11
  • 124
  • 154
0

Thanks Lalit, Adding the below line did the trick for me without have to change any java version..

compile( group: 'org.apache.hadoop', name: 'hadoop-core', version: '1.2.0' )

Alternatively, I can even add the hadoop-core jar in the class path and mention that in the gradle path

Rakshith
  • 644
  • 1
  • 8
  • 24
Kranthi
  • 109
  • 2
  • 7