When testing a specific condition / scenario, is it necessary to check both the before and after results?

Question

I'm unit testing code that deals with the delegation of time-sensitive tasks to casual workers with variable hours of availability. The general answer I'm after, however, should be independent of this context.

So, I'll have tests which follow this structure:

Test Task 1 is delegated to Bob under Specific Condition X:

1. Check Task 1 gets delegated to Frank
2. Create specific condition X
3. Test Task 1 gets delegated to Bob now, instead.

My question is, is step 1 superfluous? I already have another tests that checks step 1 alone (that under no specific conditions, the task gets delegated to Frank).

However, adding step 1 made me feel secure that my test is in fact testing the exact right thing - I verify that the result changes under condition X.

Here's another more specific example:

Test Tasks aren't delegated if no one is available

1. Create availabilities, and check tasks are delegated
2. Remove availabilities, and check tasks are NOT delegated

Again, is step 1 overkill? I already have a test to test for step 1, on it's own:

Test Tasks ARE delegated if people ARE available

1. Create availabilities, and check tasks are delegated

So, is it stupid to re-test that scenario in the previous example, just be make extra certain?

I initially (some time ago) thought it was a good idea, but reading my code now, it seems like overkill. But I want to be certain before deleting a whole bunch of code from my test suite!

Thanks!

Answer 1

For these two examples, I would not describe the first steps as additional tests, they are asserting pre-conditions. These do add value in cases where this pre-condition for the test is not obvious to the reader. This could be the case where:

• The way the tests are written means that it is not obvious to the reader that this pre-condition should hold true for the rest of the test to be valid. In this case, the pre-condition assertion helps to make the expected pre-conditions clearer to the reader. One scenario where this may happen is if your test data is being set up inside a setup method, as the separation between the data setup and the test declaration makes it harder for the reader to understand how the test works.

• You are working with highly complex code (i.e. complex mathematical algorithms), where the complex nature means it can take a while to debug test failures and understand the cause of problems. In these cases, asserting your pre-conditions can greatly help with understanding the expected behavior of the test.

Of course, as well as improving readability, you also get the benefit that if the test fails due to a pre-condition then it will be easier to understand why the test failed. However, I would not start adding pre-conditions to tests just for this reason alone - if it is the case where you require pre-condition assertions just to help you understand the cause of test failures, it is probably because your tests are overly complex.

One thing I would add, if you are using assert statements in this way, I would simply add a comment above them to make it clear to the reader that you are using this assertion to check a pre-condition.

A final note - this answer has focused solely on unit tests. As you get into different types of testing (e.g. system tests exercising the whole software stack), where tests can be more fragile, then the argument for including pre-condition assertions in your tests becomes stronger.

Answer 2

This is why I love RSpec. It helps make this kind of thing explicit.

describe 'task delegation' do
  context 'when availabilities are present' do

    before do
      # Create availabilities
    end

    it 'is assigned to person A' do
      # test the assignment'
    end

    context 'and there is a specific condition' do

      before do
        # create specific condition
      end

      it 'is assigned to person B' do
        # test the assignment
      end
    end
  end

  context 'when no availabilities are present' do
    before do
      # Do nothing
    end

    it 'is not assigned to anyone' do
      # test the lack of assignment
    end
  end
end

In writing this, we've found a couple of things. Firstly, this probably isn't a unit test in the classic test. You're testing the integration of one class (the assignment thing) with others (the availability). The only reason I could see for testing when you've first created and then deleted availability is that you don't trust something in that integration.

That could be fair enough, but it doesn't belong in an integration test.