4

Testing code that has a predefined input and output parameters is relatively easy when compared to writing tests for code that has a portion of randomness, as we must check if the random generator is biased or not.

An example of a library that uses random numbers is java.util.Collections.shuffle(List<?> list) which shuffles a collection of objects, following the http://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle.

How to write jUnit tests for code that has random output? Not only for the shuffle, but testing randomness in general.

Alexandre Santos
  • 8,170
  • 10
  • 42
  • 64
  • 1
    http://stackoverflow.com/questions/56411/how-to-test-randomness-case-in-point-shuffling – Sotirios Delimanolis Jun 25 '14 at 23:27
  • stackoverflow.com/questions/56411/… is about shuffling. Which is a subpart of this question. – Alexandre Santos Jun 25 '14 at 23:31
  • @AlexandreSantos Why doesn't the accepted answer for that question still apply? – awksp Jun 25 '14 at 23:59
  • I'm guessing there are certain range of values that output has. So what about feeding random range of inputs for `n` number of times and collect the outputs and see if each output has the same ratio? If ratios are the same, then you can tell the output is random. – Harry Cho Jun 26 '14 at 00:01

1 Answers1

1

Unless you are writing an actual random number generator or some kind of encryption library that relies on secure random number generators you don't need to check that if the random number generator is biased or not. That is the job of the author of the Random number generator.

Your example of Collections.shuffle() is also a poor example because it is a built in JDK method. There's no reason to test a built-in JDK method, the authors of Java have done that already for you plus the millions of users who have used these methods over the past 20 years. Do you also have tests to confirm System.out.print() works as expected too?

Unit tests are supposed to be deterministic. Rerunning the same test over and over again should produce the same results EVERY time. If they don't and the test fails, is it failing because the code is buggy or because the non-deterministic output produced invalid input? If when rerunning the results, the tests pass, is it because the bug was fixed or because we happened to get random input that worked?

For these reasons, your unit tests should try to mock or stub out any randomness. Perhaps write several tests that perform specific transformations of the data to test boundary conditions. For example, for shuffle() maybe have a test that reorders the elements in a specific order or sorts the elements in ascending or descending order or a test that doesn't actually do any transformation etc. These are all valid results that could happen randomly.

This way the tests all produce the same output for the same input everytime and you get the benefit of different results.

EDIT It appears you actually want to test your own random number generator.

Here are links that describe statistical tests you should do to see if it is really random (or at least as random as possible)

RANDOM.ORG statistical analysis

NIST statistical test suite

dkatzel
  • 31,188
  • 3
  • 63
  • 67
  • I didn't indicate that I wanted to test the shuffle. I trust that Sun/Oracle has done it. I just put it there as an example of a random code. However, I could have used another example, such as the random number generator for an online gaming site. For an online gaming site it is important that the random number generator is truly random. – Alexandre Santos Jun 26 '14 at 00:32
  • @AlexandreSantos added links to actually test RNGs – dkatzel Jun 26 '14 at 00:43
  • I was wondering the same thing. I know my method works because I saw the results in main of the app. Though, I am new to Junit testing and I was wondering how one can determine if a deck of cards were shuffled using a Junit test. From your answer, there seems to be no possible way. For the guy with the original questions I suggest you use your main to run the method and view the output. That is how I checked my shuffle cards method – SedJ601 Dec 02 '15 at 02:29
  • @SedrickJefferson well, you could shuffle a deck over and over again each time noting the positions of all the cards. After a sufficient number of times, you can make histograms of all the cards and where they got shuffled to. See if the cards were evenly distributed across all positions in the deck. Make sure each time you shuffle you start with a new deck in the same order. That won't test if the random number generator is secure and not predictable, but it would at least tell you the cards are getting shuffled – dkatzel Dec 02 '15 at 05:26