Testing multiple outputs with MRUnit

Question

Is there a way to test a reduce class with MRUnit that uses MultipleOutputFormat to write to multiple output files?

score 4 · Accepted Answer · answered Jan 20 '13 at 09:17

4

It looks like support for MultipleOutputs is still work in progress in the MRUnit Jira.

That being said, I found someone who implemented his own drivers subclassing MRUnit's MapReduceDriver to make it work with MultipleOutputs here, hope that helps.

answered Jan 20 '13 at 09:17

Charles Menguy

40,830
17
95
117

Thanks for finding chetan's drivers on GitHub. Have you had a chance to use them yourself by any chance? It is unclear to me how it accounts for the namedOutput and baseOutputPath parameters of MultipleOutputs.write(). – Daniel Koverman Jul 03 '13 at 20:45
Unfortunately the linked code doesn't work, it's not even valid Java code. MultiOutputMapReduceDriver.java:37 wants to use symbol "configuration", which is undefined. Also, in MultiOutputReduceDriver.java:34, the constructor used for class MockReduceContextWrapper is not defined. – gyorgyabraham Aug 12 '13 at 12:17

Géraud · Answer 2 · 2014-10-06T21:08:25.003

4

MRUnit 1.1.0 has been released in June 2014 (see http://mrunit.apache.org/)

This latest release includes support for MultipleOutputs (see https://issues.apache.org/jira/browse/MRUNIT-13)

edited Oct 06 '14 at 21:08

answered Oct 06 '14 at 19:59

Géraud

1,923
3
20
20

Testing multiple outputs with MRUnit

2 Answers2

Linked