In a job there are two input files that they are at the two different directories, In Hadoop job taking input files from multiple directories , we can read files from multiple directories. The files have the same name but they are in different name folders.
C1/part-0000
C2/part-0000
Is it possible that detecting files in map phase?
Some thing like:
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
if (First file) {
...
context.write(outputKey, outputValue);
}
} else {
//Second file
...
context.write(outputKey, outputValue);
}
}
Asked
Active
Viewed 74 times
1

user7747790
- 35
- 3
1 Answers
0
Check it in setup phase
@Override
protected void setup(Context context) throws IOException, InterruptedException {
FileSplit split = (FileSplit) context.getInputSplit();
Path path = split.getPath();
String name = path.getName();
...
Don't check it in map method for every row, because each mapper is created for 1 input split.

AdamSkywalker
- 11,408
- 3
- 38
- 76