Error in addInputPath of MapReduce Driver

Question

I am getting the error in the addInputPath method of my MapReduce Driver. The error is

"The method addInputPath(Job, Path) in the type FileInputFormat is not applicable for the arguments (JobConf, Path)"

Here is my code for the driver:

package org.myorg;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;

public class WordCount extends Configured implements Tool{
    public int run(String[] args) throws Exception
    {
          //creating a JobConf object and assigning a job name for identification purposes
          JobConf conf = new JobConf(getConf(), org.myorg.WordCount.class);
          conf.setJobName("WordCount");

          //Setting configuration object with the Data Type of output Key and Value
          conf.setOutputKeyClass(Text.class);
          conf.setOutputValueClass(IntWritable.class);

          //Providing the mapper and reducer class names
          conf.setMapperClass(WordCountMapper.class);
          conf.setReducerClass(WordCountReducer.class);

          //the hdfs input and output directory to be fetched from the command line
          **FileInputFormat.addInputPath(conf, new Path(args[0]));**
          FileOutputFormat.setOutputPath(conf, new Path(args[1]));

          JobClient.runJob(conf);
          return 0;
    }

    public static void main(String[] args) throws Exception
    {
          int res = ToolRunner.run(new Configuration(), new WordCount(),args);
          System.exit(res);
    }
}

I import the correct org.apache.hadoop.mapred.FileOutputFormat.

My WordCountMapper correctly implements Mapper.

FileOutputFormat.setOutputPath works correctly.

Why does the addInputhPaths throw an error?

score 9 · Accepted Answer · answered Dec 12 '13 at 16:54

9

The problem is that you are mixing the old API (.mapred.) and the new API (.mapreduce.). The two APIs are not compatible.

I suggest you use all the objects from the new API and not use anything from the old API. That is, don't use JobConf or JobClient. Use Job and Configuration instead. And be sure you are using Mapper, Reducer, etc. from an import that includes .mapreduce. not .mapred..

answered Dec 12 '13 at 16:54

Donald Miner

38,889
8
95
118

Thank you.There is too much confusion all around with regard to this. This clears it up. – Arun Kumar Dec 12 '13 at 20:11
For the most part I completely ignore the existence of the old API. – Donald Miner Dec 12 '13 at 21:43

Error in addInputPath of MapReduce Driver

1 Answers1