How to get the mapper output byte counter

Question

I am chaining an undetermined amount of map reduce jobs together for a parallel BFS shortest path algorithm and when the path cannot be determined, my jobs loop infinitely without producing any records. I figured the best way to check this is to get the Map Output Bytes counter that is maintained by hadoop.

How can I get access to this counter?

Answer 1

To get the map output bytes counter produced by the job, use

long outputBytes = job.getCounters().findCounter("org.apache.hadoop.mapred.Task$Counter", "MAP_OUTPUT_BYTES").getValue();

See http://lintool.github.io/Cloud9/docs/content/counters.html for more counter groups and names