17

Today I found out that the ASF retired mrunit (see https://blogs.apache.org/foundation/entry/the_apache_news_round_up85 and https://issues.apache.org/jira/browse/HADOOP-3733 and the homepage itself). Other than "inactivity" there was no reason given, so I guess there has to be an alternative?

What's supposed to be used instead of mrunit to unit test mapreduce jobs in the future?

David Ongaro
  • 3,568
  • 1
  • 24
  • 36

2 Answers2

6

I believe the reason for this retirement is the realization that Mockito already can handle what you need to unit test your mapper/reducers. All you need is to mock your Context, Counter and HBase KeyValue and you can then pass in a row (in the form of the Result object) to a mapper and check if the key/value emitted is correct.

FYI - I actually got my MR unit tests working perfectly using this approach without any assistance from MRUnit and PowerMock.

b4hand
  • 9,550
  • 4
  • 44
  • 49
SriK
  • 1,011
  • 1
  • 15
  • 29
  • 15
    This would be a better answer if there was actually some example code associated with it. – b4hand Apr 10 '17 at 18:18
  • 1
    Sorry about that. I left the company since I wrote this. Yeah, I should have given sample code. It was fairly tricky to setup if you are a newbie to mockito unit testing in java... – SriK Aug 15 '18 at 21:49
3

Not exactly a successor, but I tried and tested a work-around with a bit of research. It is actually based on the Minicluster module of hadoop.

To find out how to use it, details can be found in this link here. And a github demo project exists here.

Saif Asif
  • 5,516
  • 3
  • 31
  • 48
  • 2
    One upvote for the research. Indeed we're already using `MiniDFSCluster` in one instance and surely could use it more. But it's not the answer I'm looking for, because the MiniCluster is more of a functional level test. We also want to cover the unit level aspect (so basically just the map/reduce methods) – David Ongaro Sep 01 '16 at 05:58
  • @DavidOngaro Correct, I am also thinking to expand using MiniDFSCluster and include some unit testing in it as well. – Saif Asif Sep 01 '16 at 06:59
  • detail link expired – cdarlint Sep 06 '21 at 13:58