2

I have a unit test that creates an error condition. Normally, the class under test writes this error to the logs (using log4j in this case, but I don't think that matters). I can change the log level temporarily, using

Logger targetLogger = Logger.getLogger(ClassUnderTest.class);
Level oldLvl = targetLogger.getLevel();
targetLogger.setLevel(Level.FATAL);

theTestObject.doABadThing();

assertTrue(theTestObject.hadAnError());

targetLogger.setLevel(oldLvl);

but that also means that if an unrelated / unintended error occurs during testing, I won't see that information in the logs either.

Is there a best practice or common pattern I'm supposed to use here? I don't like prodding the log levels if I can help it, but I also don't like having a bunch of ERROR noise in the test output, which could scare future developers.

Coderer
  • 25,844
  • 28
  • 99
  • 154

2 Answers2

1

If your logging layer permits, it is a good practice to make an assertion on the error message. You can do it by implementing your own logger that just asserts on the message (without output), or by using a memory-buffer logger and then check on the contents of the log buffer.

Under no circumstances should the error message end up in the unit-test execution log. This will cause people to get used to errors in the log and mask other errors. In short, your options are:

  1. Most preferred: Catch the message in the harness and assert on it.
  2. Somewhat OK: Raise the level and ignore the message.
  3. Not OK: Don't do anything and let the log message reach stderr/syslog.
thiton
  • 35,651
  • 4
  • 70
  • 100
  • OK, I think I've seen somebody do #1 before, but I'm going to have to dig around a bit for sample code. Like I said, I'm using Log4J, so if you *happen* to have a handy link to an example, it'd be most appreciated! – Coderer Feb 12 '13 at 11:23
  • Ah, I've just found [this](http://stackoverflow.com/questions/1827677/how-to-do-a-junit-assert-on-a-message-in-a-logger) which looks like it should do the trick. Thanks! – Coderer Feb 12 '13 at 11:28
0

The way I approach this assuming an XUnit style of unit testing (Junit,Pyunit, etc) 

@Test(expected = MyException)  
 foo_1() throws Exception
{
    theTestObject.doABadThing(); //MyException here
}

The issue with doing logging is that someone needs to go and actually parse the log file, this is time consuming and error prone. However the test will pass above if MyException is generated and fail if it isn't. This in turn allows you to fail the build automatically instead of hoping the tester read the logs correctly.

Woot4Moo
  • 23,987
  • 16
  • 94
  • 151
  • Thanks, I'm familiar with this convention, but in my particular case scanning the logs is actually much simpler than this type of test. It would work if I were testing at a lower level, but unfortunately I'm stuck with basically integration testing, given the application I'm writing a plugin for. – Coderer Feb 12 '13 at 16:30