Implementation of oder by in hadoop using Map Reduce

Question

I want to implement order by like clause in my Map Reduce program.

My input looks like this

1,Subhradip Bose,1

2,Prajakta Bose,2

2,Anup Singh,3

3,Mahesh Gurav,4

I have written a map function which looks like

protected void map(LongWritable key, Text value, Context context)
        throws IOException, InterruptedException {
    String[] currentLine = value.toString().split(",");
    context.write(new Text(currentLine[1]), new Text(currentLine[1]));
}

But I am getting the output like this

Subhradip Bose

Prajakta Bose

Anup Singh

Mahesh Gurav

I want the output like the following

Anup Singh

Mahesh Gurav

Prajakta Bose 

Subhradip Bose

which is like clause, 1st column or 3rd one? – Sandy Dec 23 '14 at 15:02 — Sandy, Dec 23 '14 at 15:02

score 0 · Accepted Answer · edited May 23 '17 at 11:49

I think you are not using reducer and the mapper output is final output

If you introduce reducers then shuffling and sorting would happen and you would get desired output.

Please refer following question what-is-the-purpose-of-shuffling-and-sorting-phase-in-the-reducer

sample Reducer implementation :

public class Reduce     
 extends Reducer<Text, Text,Text, Text> {


public void reduce(Text key, Iterable<Text> values, Context context) 
 throws IOException,InterruptedException {

  Iterator<Text> it = values.iterator();
  if(it.hasNext()){ // you can use loop here
      Text name = it.next();
      context.write(key, name);
  }

 }

}

welcome, may be you would like to accept the answer :) – Sandy Dec 23 '14 at 17:03 — Sandy, Dec 23 '14 at 17:03

Implementation of oder by in hadoop using Map Reduce

1 Answers1