Compilation errors while compiling WordCount.java

Question

I keep getting an error which says 'cannot find symbol' while compiling WordCount.java. I tried importing the other libraries, but it only adds up to the list of errors. Am I missing some component that needs to be added?

  javac -classpath /Documents/hadoop/hadoop-1.2.1/hadoop-core-1.2.1.jar:/Documents/hadoop/hadoop-1.2.1/lib/commons-cli-1.2.jar -d wordcount_classes WordCount.java


    WordCount.java:4: error: package org.apache.hadoop.fs does not exist
    import org.apache.hadoop.fs.Path;
                               ^
    WordCount.java:5: error: package org.apache.hadoop.io does not exist
    import org.apache.hadoop.io.*;
    ^
    WordCount.java:6: error: package org.apache.hadoop.mapred does not exist
    import org.apache.hadoop.mapred.*;
    ^
    WordCount.java:8: error: package org.apache.hadoop.mapred.lib.input does not exist
    import org.apache.hadoop.mapred.lib.input.FileInputFormat;
                                             ^
    WordCount.java:9: error: package org.apache.hadoop.mapred.lib.output does not exist
    import org.apache.hadoop.mapred.lib.output.FileOutputFormat;
                                          ^
    WordCount.java:10: error: package org.apache.hadoop.mapred.lib.input does not exist
    import org.apache.hadoop.mapred.lib.input.TextInputFormat;
                                         ^
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,    Text, Text, IntWritable> {
                                        ^
    symbol:   class MapReduceBase
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,   Text, Text, IntWritable> {
                                                                 ^
    symbol:   class Mapper
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,     Text, Text, IntWritable> {
                                                                        ^
    symbol:   class LongWritable
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                      ^
    symbol:   class Text
    location: class WordCount 
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,     Text, Text, IntWritable> {
                                                                                             ^
    symbol:   class Text
    location: class WordCount
WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                                  ^
    symbol:   class IntWritable
    location: class WordCount
    WordCount.java:16: error: cannot find symbol
          private final static IntWritable one = new IntWritable(1);
                               ^
    symbol:   class IntWritable
    location: class Map
    WordCount.java:17: error: cannot find symbol
          private Text word = new Text();
                  ^
    symbol:   class Text
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                          ^
    symbol:   class LongWritable
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable>   output, Reporter reporter) throws IOException {
                                            ^
    symbol:   class Text
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable>     output, Reporter reporter) throws IOException {
                                                        ^
    symbol:   class OutputCollector
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                        ^
  symbol:   class Text
  location: class Map
WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                              ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                   ^
  symbol:   class Reporter
  location: class Map
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                           ^
  symbol:   class MapReduceBase
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                    ^
  symbol:   class Reducer
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                            ^
  symbol:   class Text
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                  ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                               ^
  symbol:   class Text
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                                     ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                             ^
  symbol:   class Text
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                     ^
  symbol:   class OutputCollector
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                     ^
  symbol:   class Text
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                           ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                                ^
  symbol:   class Reporter
  location: class Reduce
WordCount.java:16: error: cannot find symbol
          private final static IntWritable one = new IntWritable(1);
                                                     ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:17: error: cannot find symbol
          private Text word = new Text();
                                  ^
  symbol:   class Text
  location: class Map
WordCount.java:34: error: cannot find symbol
            output.collect(key, new IntWritable(sum));
                                    ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:39: error: cannot find symbol
          JobConf conf = new JobConf(WordCount.class);
          ^
  symbol:   class JobConf
  location: class WordCount
WordCount.java:39: error: cannot find symbol
          JobConf conf = new JobConf(WordCount.class);
                             ^
  symbol:   class JobConf
  location: class WordCount
WordCount.java:42: error: cannot find symbol
          conf.setOutputKeyClass(Text.class);
                                 ^
  symbol:   class Text
  location: class WordCount
WordCount.java:43: error: cannot find symbol
          conf.setOutputValueClass(IntWritable.class);
                                   ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:49: error: cannot find symbol
          conf.setInputFormat(TextInputFormat.class);
                              ^
  symbol:   class TextInputFormat
  location: class WordCount
WordCount.java:50: error: cannot find symbol
          conf.setOutputFormat(TextOutputFormat.class);
                               ^
  symbol:   class TextOutputFormat
  location: class WordCount
WordCount.java:52: error: cannot find symbol
          FileInputFormat.setInputPaths(conf, new Path(args[0]));
                                                  ^
  symbol:   class Path
  location: class WordCount
WordCount.java:52: error: cannot find symbol
          FileInputFormat.setInputPaths(conf, new Path(args[0]));
          ^
  symbol:   variable FileInputFormat
  location: class WordCount
WordCount.java:53: error: cannot find symbol
          FileOutputFormat.setOutputPath(conf, new Path(args[1]));
                                                   ^
  symbol:   class Path
  location: class WordCount
WordCount.java:53: error: cannot find symbol
          FileOutputFormat.setOutputPath(conf, new Path(args[1]));
          ^
  symbol:   variable FileOutputFormat
  location: class WordCount
WordCount.java:55: error: cannot find symbol
      JobClient.runJob(conf);
      ^
  symbol:   variable JobClient
  location: class WordCount
46 errors

Tip: Try an IDE such as Eclipse or NetBeans, it makes development a million times easier. — Kyranstar, May 03 '14 at 02:47

score 0 · Answer 1 · edited May 03 '14 at 03:07

0

You can try this one:

javac -classpath hadoop-core-1.2.1.jar:lib/commons-cli-1.2.jar -d [path where u want to put the result] [path]/WordCount.java

edited May 03 '14 at 03:07

Rob Watts

6,866
3
39
58

answered May 03 '14 at 02:43

Joyce

9
1

score 0 · Answer 2 · answered Dec 27 '16 at 08:57

It seems your classpath doesn't include all required libs and jars.

For hadoop 2.7 you can easily run:

$hadoop classpath

This will return the full list of libs and jars which you need to include in your classpath.

or simpler you can just use the above command when you are compiling your code

$  javac -classpath $(hadoop classpath) -d wordcount_classes WordCount.java

score 0 · Answer 3 · answered Apr 02 '17 at 19:45

It is because the necessary packages are not included in javac classpath.

For hadoop 2.7.3, adding 3 packages works for me:

javac -classpath $HADOOP_HOME$/share/hadoop/common/hadoop-common-2.7.3.jar:$HADOOP_HOME$/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.3.jar:$HADOOP_HOME$/share/hadoop/common/lib/commons-cli-1.2.jar
-d wordcount_classes WordCount.java

J. Doe · Answer 4 · 2017-09-20T09:22:41.043

Some of the other answers cut to the chase and give pragmatic, quick solutions - which will work - but the official Hadoop Wordcount tutorial recommends that you call javac from bin/hadoop. This way, Hadoop will take care of the classpath conflicts for its own libraries.

The above-mentioned tutorial could however mislead you to point HADOOP_CLASSPATH to the wrong lib directory (there are distinct ones for the JDK and the JRE) thus missing tools.jar.*

My solution is to proceed as follow:

$ export HADOOP_PREFIX="${absolute/path/to/your/hadoop/binaries}"
$ export JDK_HOME="${absolute/path/to/your/jdk}"  

$ export JAVA_HOME="${JDK_HOME}/jre"  
$ export PATH="${JAVA_HOME}/bin:${PATH}"  
$ export HADOOP_CLASSPATH="${JDK_HOME}/lib/tools.jar"  

$ cd $HADOOP_PREFIX
$ bin/hadoop com.sun.tools.javac.Main ./path/to/WordCount.java

Note that the actual values to assign to JAVA_HOME and HADOOP_PREFIX must point to the locations where those tools are actually installed on your machine.

* it would cause Error: Could not find or load main class com.sun.tools.javac.Main at compile-time

Compilation errors while compiling WordCount.java

4 Answers4

Linked