Stack Overflow
Stack Overflow

Questions tagged [mrunit]

Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.

Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.

64 questions
17
votes
2 answers

What's the successor of mrunit?

Today I found out that the ASF retired mrunit (see https://blogs.apache.org/foundation/entry/the_apache_news_round_up85 and https://issues.apache.org/jira/browse/HADOOP-3733 and the homepage itself). Other than "inactivity" there was no reason…
David Ongaro
  • 3,568
  • 1
  • 24
  • 36
8
votes
0 answers

MRunit: how to test custom sortcomparator?

I'm trying to use MRunit to test my sortComparatorClass. It seems MRunit should be able to do this with the setKeyOrderComparator method, but when I run the mapReduceDriver it is not calling the compare() method of SortComparator class. Pretty sure…
alexP_Keaton
  • 349
  • 5
  • 18
7
votes
1 answer

Hive unit testing on windows without hadoop setup

I am looking to see if there is a way to run Hive unit tests (i.e unit testing the hive queries) in Windows without having a hadoop/hive/cygwin setup. Came across these open source tools (Hive_runner, HiveTest), tried running hiverunner in Eclipse…
Srivatsan Nallazhagappan
  • 297
  • 1
  • 4
  • 14
6
votes
2 answers

Testing multiple outputs with MRUnit

Is there a way to test a reduce class with MRUnit that uses MultipleOutputFormat to write to multiple output files?
David Parks
  • 30,789
  • 47
  • 185
  • 328
5
votes
1 answer

Why mrunit 1.1.0 jar is not available in Maven central repository

I tried to test my MapReduce 2 job with mrunit in a maven project. I searched from Maven central repo and found the result: http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.apache.mrunit%22%20AND%20a%3A%22mrunit%22 In the link above, the…
Rui
  • 3,454
  • 6
  • 37
  • 70
5
votes
1 answer

MRunit dependency latest in cloudera repository

I could not find the latest mrunit(1.1.0) in Cloudera repository. The one available is 0.8.0-incubating. Following is my pom:
Dutta
  • 663
  • 2
  • 11
  • 30
5
votes
1 answer

How to do Mapper testing using MRUnit Test?

I am new to Hadoop. I want to test my mapper part alone using MRUnit Test. I have tried a lot. But i dont know how to solve the following error- "The method setMapper(Mapper) in the type MapDriver is not applicable for the arguments…
Karthick
  • 97
  • 1
  • 1
  • 7
4
votes
4 answers

MRUnit with Avro NullPointerException in Serialization

I'm trying to test a Hadoop .mapreduce Avro job using MRUnit. I am receiving a NullPointerException as seen below. I've attached a portion of the pom and source code. Any assistance would be appreciated. Thanks The error I'm getting is :…
Paul Mazzuca
  • 567
  • 1
  • 5
  • 19
3
votes
0 answers

MRUnit for Parquet

I am trying to test my driver with MRUnit and my MapDriver format is MapDriver I have found the correct serialization for KeyValue but cannot find one for Group and I get the following error. …
abtomb
  • 31
  • 2
3
votes
1 answer

MRUnit test giving NULLPOINTER exception while writing to HDFS using MULTIPLEOUTPUTS

I currently have a mapReduce program that send data to hdfs with different file name.So in my reducer I am using MultipleOutputs to write to different files in HDFS (Full Reducer code below). I would like to test my code using mrunit and below is my…
himaja
  • 33
  • 4
3
votes
1 answer

Received unexpected output in MRUnit

I'm getting the following MRUnit error: ERROR mrunit.TestDriver: Received unexpected output (60, mrdp.MyCustomClass@73207f36) ERROR mrunit.TestDriver: Missing expected output (60, mrdp.MyCustomClass@6f73cf45) at position 0 I've created a…
guzu92
  • 737
  • 1
  • 12
  • 28
3
votes
2 answers

MRUnit - Not Working as expected

I am having only mapper class that creates files for bulk loading into HBase and I have written a MRUnit for Unit testing. Though the expected result and the retruned results are same, the MRUnit is failing with the message "Missing expected…
Suresh
  • 186
  • 2
  • 14
2
votes
1 answer

MRUnit test for testing hadoop map program error

I'm trying to unit test a Mapper program using MRUnit (from Hadoop:The definitive guide book, page 153, section: Writing a unit test with MRUnit: Mapper). I'm using intellij Idea and it is showing an error in method new…
Liberty_bird
  • 45
  • 6
2
votes
1 answer

MRUnit Context object returning empty configuration

Need one help with MRUnit. I am adding my config file to MapReduceDriver as below. conf = mapReduceDriver.getConfiguration(); conf.addResource("path_to_config.xml"); When reducer class is trying to access the the property in setUp() mehtod, its not…
SurjanSRawat
  • 489
  • 1
  • 6
  • 20
2
votes
0 answers

How to write mrunit test case for (multiple mappers and single reducer) MMR Driver

I am new to BigData I have written a MR program and trying to write test cases for it using MRUnit from https://dzone.com/articles/testing-mapreduce-mrunit But My MR program has 2 mappers and 1 reducer so i am not able to create a driver object…
HkFreaKuser1673718
  • 759
  • 4
  • 13
  • 31
1
2 3 4 5