Hi I am running mapreduce code on eclipse using Cygwin. I am able to successfully run wordcount program in this environment. But for my new code I am getting below exception. My program does not have any reducer job/class. I also debug the code in eclipse. All mapper jobs running successfully and writing output in context. After that exception is thrown. Temporary output folders are created but no final output. Please help me to solve this problem.
Thanks
java.lang.RuntimeException: java.lang.InstantiationException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:530)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:410)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
Caused by: java.lang.InstantiationException
at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113
... 3 more
Please find my main function below.
public static void main(String[] args) throws Exception {
if (args.length < 5) {
System.out.println("Arguments: [model] [dictionnary] [document frequency] [tweet file] [output directory]");
return;
}
String modelPath = args[0];
String dictionaryPath = args[1];
String documentFrequencyPath = args[2];
String tweetsPath = args[3];
String outputPath = args[4];
Configuration conf = new Configuration();
conf.setStrings(Classifier.MODEL_PATH_CONF, modelPath);
conf.setStrings(Classifier.DICTIONARY_PATH_CONF, dictionaryPath);
conf.setStrings(Classifier.DOCUMENT_FREQUENCY_PATH_CONF, documentFrequencyPath);
// do not create a new jvm for each task
conf.setLong("mapred.job.reuse.jvm.num.tasks", -1);
Job job = new Job(conf, "classifier");
job.setJarByClass(MapReduceClassifier.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setMapperClass(ClassifierMap.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
FileInputFormat.addInputPath(job, new Path(tweetsPath));
FileOutputFormat.setOutputPath(job, new Path(outputPath));
job.waitForCompletion(true);
}
