11

Is there a flexible framework for fuzzy testing in Java?

The Wikipedia article on Fuzz testing defines fuzz testing as follows:

Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.

I know there is such a framework for Erlang. Any hints are welcome.

Oliver
  • 3,815
  • 8
  • 35
  • 63
  • possible duplicate: http://stackoverflow.com/questions/1099124/looking-for-a-java-fuzzing-library-or-fuzzer-tool-with-a-good-api – Dan Temple Mar 13 '14 at 15:14
  • Maybe you give this one a try: https://code.google.com/p/terot/ or this one: http://people.csail.mit.edu/akiezun/jfuzz/ – mhafellner Mar 13 '14 at 15:16
  • Simliar: http://stackoverflow.com/questions/32458/random-data-in-unit-tests . There is also http://pitest.org but that actually modifies your code in order to "test" your tests so not really the same as what you want. I've used http://github.com/rickynils/scalacheck for randomising data and it works pretty good. – Peter Svensson Mar 13 '14 at 15:19
  • Thanks for all the suggestions. All of them seems to be not very actively developed but I will try them out. – Oliver Mar 20 '14 at 21:37
  • One I wrote recently: https://github.com/cretz/javan-warty-pig – Chad Retz Jun 14 '18 at 14:10
  • 1
    @Oliver did you find one that suited your needs? – payne May 25 '20 at 20:25
  • No, but I haven't searched for such a library for a long time. I started to use mutation testing via PIT (https://pitest.org/). It helped me a lot to improve my code and my test and also the way how to structure code to be able to test it in a better way. – Oliver May 26 '20 at 21:30
  • Ping @payne. I forgot to mention you in my answer. – Oliver May 27 '20 at 11:02
  • Thanks! I just happen to have discovered this library yesterday, haha. – payne May 27 '20 at 12:57

2 Answers2

3

You may want to look into JQF:

https://github.com/rohanpadhye/jqf

Flow
  • 23,572
  • 15
  • 99
  • 156
1

Looks like there is a Java port of Quickcheck.

The goal of QuickCheck is to replace manually picked values with generated values. A QuickCheck-based test tries to cover the laws of a domain whereas classical testing can only test the validity for distinct values.

Basically, QuickCheck is about generators of data. The QuickCheck runner method is just a fancy for loop implementation. QuickCheck can help in scenarios where whole classes of test cases have to be tested and it is not feasible to write tests for all distinct test scenarios.

Community
  • 1
  • 1
matt b
  • 138,234
  • 66
  • 282
  • 345
  • 8
    Quickcheck is actually the *opposite* of a fuzzy testing. Quickcheck allows you to define a range of different inputs that you can test to see if they _conform_ to your business rules. Fuzzy testing is the opposite idea: find input that _breaks_ your system. – Marco Nov 12 '16 at 21:55
  • @Marco No, see the definition of Quickcheck above. Domains and rules are very generally defined. In fact, it uses generators to try edge cases early to try to test inputs most likely to break the broad rules supplied. – brianary Sep 12 '21 at 14:54
  • @Marco This is a rumination on the topics that doesn't really seem to conclude definitively. https://hypothesis.works/articles/what-is-property-based-testing/ – brianary Sep 12 '21 at 15:09