Name clash of getPartition of type Partitioner has the same erasure of type main class in MapReduce, Hadoop

Question

I was trying to write a code that I can customize the Input will go to the reducer according to the length of the character using implementing to the Partition where default Mapper and Reducer, but the following error is coming. I will be thankful to someone who will help me.

Error in int setNumRedTasks):

Name clash: The method getPartition(Object, Object, int) of type MyPartitioner has the same erasure as getPartition(K2, V2, int) of type Partitioner but does not override it

Code:

package partition;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.Partitioner;

public abstract class MyPartitioner implements Partitioner<Text, IntWritable>{

    @Override
    public void configure(JobConf arg0) {
        // TODO Auto-generated method stub

    }
    // @Override
    public int getPartition(Object key, Object value, int setNumRedTasks) {
        String s = key.toString();
        if(s.length()==1)
        {
            return 0;
        }

        if(s.length()==2)
        {
            return 1;
        }

        if(s.length()==3)
        {
            return 2;
        }
        else
            return 3;


    }
}

score 0 · Accepted Answer · answered Aug 30 '17 at 07:04

Your getPartition method signature is wrong, it need to be:

public int getPartition(Text key, IntWritable value, int setNumRedTasks) {
    ... Code goes here
}

This SO answer explains what the erasure error means: Method has the same erasure as another method in type

Effectively, because you used Object instead of the generic types it can't work out which version to use, they are equivalent.

Name clash of getPartition of type Partitioner has the same erasure of type main class in MapReduce, Hadoop

1 Answers1