1

In a job there are two input files that they are at the two different directories, In Hadoop job taking input files from multiple directories , we can read files from multiple directories. The files have the same name but they are in different name folders. C1/part-0000 C2/part-0000 Is it possible that detecting files in map phase?
Some thing like: public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { if (First file) { ... context.write(outputKey, outputValue); } } else { //Second file ... context.write(outputKey, outputValue); } }

1 Answers1

0

Check it in setup phase

@Override
protected void setup(Context context) throws IOException, InterruptedException {
    FileSplit split = (FileSplit) context.getInputSplit();
    Path path = split.getPath();
    String name = path.getName();
    ...

Don't check it in map method for every row, because each mapper is created for 1 input split.

AdamSkywalker
  • 11,408
  • 3
  • 38
  • 76