how to access Mapper Counter value in a Reducer?

Question

I want to acces the myCounter.my value in reducer :

public static class Map extends Mapper<LongWritable, Text, ImmutableBytesWritable, ImmutableBytesWritable>
{
    public static enum myCounter{my};

    @Override
    public void map(LongWritable key, Text value, Context context) 
    {
        context.getCounter(myCounter.my).increment(1);
        context.write( new ImmutableBytesWritable ( ),new ImmutableBytesWritable() );
    }
}


public static class Reduce extends Reducer<ImmutableBytesWritable, ImmutableBytesWritable, Text, Text>
{
    @Override
    public void reduce(ImmutableBytesWritable key,Iterable<ImmutableBytesWritable> result,Context context)
    {

    }
}

Accessing a mapper's counter from a reducer(for old API is given ) how to make it work for new API ?

Or

I want to know the total number of mapper output ? Is there any better way ? (i am not able to access counter in Reducer:

Group Name->org.apache.hadoop.mapred.Task$Counter Counter Name->MAP_OUTPUT_RECORDS)

Thanks

[example](http://lintool.github.com/Cloud9/docs/content/counters.html) — faizan, Sep 25 '12 at 10:33
@faizan: ty but as you can see the exanple is abt accessing counter via Job ,,either in main(String args[]) or other functions only after the Job has finished ruining ,**where as I want to access it in Reduce**. so still my question is not solved . — saurabh shashank, Sep 25 '12 at 10:42

score 2 · Answer 1 · answered Mar 18 '17 at 22:04

You to make it work for new API by accessing the counters via job object.

Configuration conf = context.getConfiguration();
Cluster cluster = new Cluster(conf);
Job currentJob = cluster.getJob(context.getJobID());
long val=currentJob.getCounters().findCounter(myCounter.my).getValue();

score 0 · Answer 2 · edited May 23 '17 at 11:46

0

You can access your counter value in reducer using the same code

Counter counter = context.getCounter( myCounter.my );
        counter.getValue();

also see

edited May 23 '17 at 11:46

Community

1
1

answered Sep 25 '12 at 08:38

faizan

680
1
6
15

Its Not working .counter.getValue will give zero since ,the context(org.apache.hadoop.mapreduce.Reducer.Context ) in Reducer is different then context(org.apache.hadoop.mapreduce.Reducer.Context ) of Mapper . – saurabh shashank Sep 25 '12 at 08:44
as far as i know the context gives you access to same Global counter. – faizan Sep 25 '12 at 08:46
so it does not affect whether you are accessing context from setup or map or reduce method [link](http://diveintodata.org/2011/03/15/an-example-of-hadoop-mapreduce-counter/) – faizan Sep 25 '12 at 08:47
At first even ,i tried that but its not working , its returning zero , so i came to that conclusion . – saurabh shashank Sep 25 '12 at 08:58
I think you will get your answer [here](http://stackoverflow.com/questions/5450290/accessing-a-mappers-counter-from-a-reducer) – faizan Sep 25 '12 at 12:00
I have shared this link in my qus . Its old API i am using new API . **How to implement that solution in new API ?** – saurabh shashank Sep 25 '12 at 12:04

score -2 · Answer 3 · answered Sep 25 '12 at 06:18

-2

You can access as follows:

Map.myCounter.my // static fields you can access by it's class name.

answered Sep 25 '12 at 06:18

Azodious

13,752
1
36
71

In hadoop Map-Reduce framework it doesn't work like this .My Question is specific for Map_reduce Framework . – saurabh shashank Sep 25 '12 at 08:10

how to access Mapper Counter value in a Reducer?

3 Answers3

Linked