How to Sort on values in Hadoop?

Asked Feb 13 '17 at 14:00

Active Feb 14 '17 at 11:38

Viewed 524 times

I have a history file, that consists of multiple (Topic Pages) each page has 14 lines, from which I need to take an ID and then I count the number of appearances of this particular document, that how many times it appears in this history file. But then I need to display the sorted output with highest number of appearances and then by TOPIC PAGE ID.

In mapper I am just taking the TOPIC key, and using IntWritable writing one against every entry.

and Then in Reducer I just sums it up.

Can't do it using secondary sort, as I get the total count for each TOPIC Page after reduce function has been called.

Output would be like:

TopicID Appearances
987634 89
678945 87
378956 76

asked Feb 13 '17 at 14:00

Asad

1

Apache pig makes this trivial. Have you tried it? – OneCricketeer Feb 13 '17 at 14:13
1

Possible duplicate of [how to sort word count by value in hadoop?](http://stackoverflow.com/questions/18403857/how-to-sort-word-count-by-value-in-hadoop) – Serhiy Feb 13 '17 at 15:15
I want to avoid the use of two jobs. – Asad Feb 13 '17 at 16:24
Apache pig is other framework, I want it with the Hadoop. – Asad Feb 13 '17 at 16:25
could you provide a sample input and output for better understanding? – srikanth Feb 14 '17 at 11:22

How to Sort on values in Hadoop?

0 Answers0