5

I just read question that answered what were desirable features of unit tests, but what should be avoided? What makes a unit test "bad"?

What are the worst unit tests you've seen? (For example. I remember a developer telling me he once found a test suite with plenty of methods but completely devoid of any asserts).

I'm particularly interested in slightly more subtle and specific problems with unit tests e.g. suppose you have a test-suite that runs quickly with good coverage, what problems could it still have?

Community
  • 1
  • 1
Adrian Mouat
  • 44,585
  • 16
  • 110
  • 102
  • This is a fairly subjective question - are you sure it belongs on stack overflow? – Alex Reece Oct 12 '10 at 11:14
  • I don't think it's that subjective - most people seem to agree on what bad code is. – Adrian Mouat Oct 12 '10 at 15:35
  • To give some more background, it occurred when I wanted to give people an example of a bad unit test and found it harder than I expected. It seems easier to come up with examples of good (or reasonable) tests, possibly as examples tend to be necessarily succinct. – Adrian Mouat Oct 12 '10 at 15:40
  • Given a piece of code-under-test, just showing a piece of bad unit test is not very meaningful (barring simple illogicality) unless a better alternative is shown for comparison. It should be clearly listed the conditions that should have been tested for, and how the good-test/bad-test covers or misses each one of them. There are also good/bad attributes related to the *manageability* of unit tests, however, nowadays this concern is being hyped over the actual need to improve quality. – rwong Oct 20 '10 at 12:45
  • @rwong, Fair point, feel free to show a corrected example alongside a bad example. – Adrian Mouat Oct 21 '10 at 14:00

6 Answers6

9

  • Tests with external dependencies (DB, file, server, time...)

  • Tests that depend on each other

  • Tests that verify the implementation rather than the behavior

  • Tests that are so slow that no one execute them

  • Tests that test too much things

There is also the TDD anti-patterns.

Bert F
  • 85,407
  • 12
  • 106
  • 123
philant
  • 34,748
  • 11
  • 69
  • 112
  • Good link! I'd forgotten about that post, but I found that I'd actually commented on it. It's a small world... – Adrian Mouat Oct 13 '10 at 08:37
  • I honestly don't mind a test having external dependencies as long as they are not significantly slowing down the test and they work for everyone. Often it's more work than it's worth to mock them out (I'm thinking specifically of DBs and files here). – Adrian Mouat Oct 13 '10 at 08:41
  • Some dependencies are annoying, think about a missing configuration or data file, or a test attempting to create a file in directory without write permissions... – philant Oct 13 '10 at 19:27
  • I'm also wondering about the implementation vs behaviour point, but I don't have enough room to discuss it here. Perhaps this is more sbjective than I first thought... – Adrian Mouat Oct 13 '10 at 20:34
  • Tests that verify the implementation break when the implementation changes. Tests must assert what is done and not how it is done. Tentative example: test of some data storage class should use the method to retrieve the data, not directly peek in the storage array because it will break if the array is replaced with a vector. – philant Oct 15 '10 at 13:36
4

For me readability is King when it comes to unit tests. If I can't read and understand the test in 2 seconds, there's probably something wrong. Any test that's more than 5 lines long had better have a pretty good excuse.

Sometimes people take the refactoring too far and I have to look at various helper classes or parent classes to find out what exactly is being tested. Always keep readability in mind when refactoring tests. sometimes it's better to leave a bit of duplication in there if it means the test is clearer.

Jonny Cundall
  • 2,552
  • 1
  • 21
  • 33
2

Tests that are brittle usually have an unacceptable maintenance overhead which will evenutally lead to the tests not being updated, remaining in a broken state, and not being run since they are out of synch with the source code.

Brittle tests usually have dependencies on file system, registry keys, databases etc... These are fine with Integration and System tests but sometimes I see tests masquerading (spelling?) as unit tests with these properties and that's usually a problem.

Ben Cawley
  • 1,606
  • 17
  • 29
1

The best unit tests were simple to read and understand. Quick to execute. Tested specific functionality, well refactored and were maintained.

The worst were not the above.

Preet Sangha
  • 64,563
  • 18
  • 145
  • 216
1

In the spirit of CW, I knew this would come in handy one day.

Also you did ask for a specific one :) See the MAGIC block below

@Test
      public void testCheckForDuplicateCustomer() {
            //List<CustomerSearch> customerInfo = null;
            String customerName = null;
            boolean status = false;
            try {
                  status = custSearchService.checkForDuplicateCustomer(customerName);

/*************/ MAGIC BEGINS HERE
                      if(status){
                            assertEquals(true, status);
                      } else {
                            assertEquals(false, status);
                      }
                      /**************/ MAGIC ENDS HERE
                } catch (Exception e) {
                      //fail();
                }
          }
JoseK
  • 31,141
  • 14
  • 104
  • 131
  • That made me smile, so I upvoted. However, I'm really looking for examples of code that isn't necessarily bone-headed but doesn't make a good unit test. – Adrian Mouat Oct 12 '10 at 15:41
0

Tests that do not test enough or anything at all.

For example, just checking return value from the method is not good enough if related output parameters or object data are not validated upon method execution.

Tests are running fine, coverage is high up but you are not validating anything really...

ratkok
  • 739
  • 9
  • 15