Should it be "Arrange-Assert-Act-Assert"?

Question

Regarding the classic test pattern of Arrange-Act-Assert, I frequently find myself adding a counter-assertion that precedes Act. This way I know that the passing assertion is really passing as the result of the action.

I think of it as analogous to the red in red-green-refactor, where only if I've seen the red bar in the course of my testing do I know that the green bar means I've written code that makes a difference. If I write a passing test, then any code will satisfy it; similarly, with respect to Arrange-Assert-Act-Assert, if my first assertion fails, I know that any Act would have passed the final Assert - so that it wasn't actually verifying anything about the Act.

Do your tests follow this pattern? Why or why not?

Update Clarification: the initial assertion is essentially the opposite of the final assertion. It's not an assertion that Arrange worked; it's an assertion that Act hasn't yet worked.

Answer 1

This is not the most common thing to do, but still common enough to have its own name. This technique is called Guard Assertion. You can find a detailed description of it on page 490 in the excellent book xUnit Test Patterns by Gerard Meszaros (highly recommended).

Normally, I don't use this pattern myself, since I find it more correct to write a specific test that validates whatever precondition I feel the need to ensure. Such a test should always fail if the precondition fails, and this means that I don't need it embedded in all the other tests. This gives a better isolation of concerns, since one test case only verifies one thing.

There may be many preconditions that need to be satisfied for a given test case, so you may need more than one Guard Assertion. Instead of repeating those in all tests, having one (and one only) test for each precondition keeps your test code more mantainable, since you will have less repetition that way.

Answer 2

It could also be specified as Arrange-Assume-Act-Assert.

There is a technical handle for this in NUnit, as in the example here: http://nunit.org/index.php?p=theory&r=2.5.7

Answer 3

Here's an example.

public void testEncompass() throws Exception {
    Range range = new Range(0, 5);
    assertFalse(range.includes(7));
    range.encompass(7);
    assertTrue(range.includes(7));
}

It could be that I wrote Range.includes() to simply return true. I didn't, but I can imagine that I might have. Or I could have written it wrong in any number of other ways. I would hope and expect that with TDD I actually got it right - that includes() just works - but maybe I didn't. So the first assertion is a sanity check, to ensure that the second assertion is really meaningful.

Read by itself, assertTrue(range.includes(7)); is saying: "assert that the modified range includes 7". Read in the context of the first assertion, it's saying: "assert that invoking encompass() causes it to include 7. And since encompass is the unit we're testing, I think that's of some (small) value.

I'm accepting my own answer; a lot of the others misconstrued my question to be about testing the setup. I think this is slightly different.

Answer 4

An Arrange-Assert-Act-Assert test can always be refactored into two tests:

1. Arrange-Assert

and

2. Arrange-Act-Assert

The first test will only assert on that which was set up in the Arrange phase, and the second test will only assert for that which happened in the Act phase.

This has the benefit of giving more precise feedback on whether it's the Arrange or the Act phase that failed, while in the original Arrange-Assert-Act-Assert these are conflated and you would have to dig deeper and examine exactly what assertion failed and why it failed in order to know if it was the Arrange or Act that failed.

It also satisfies the intention of unit testing better, as you are separating your test into smaller independent units.

Answer 5

I am now doing this. A-A-A-A of a different kind

Arrange - setup
Act - what is being tested
Assemble - what is optionally needed to perform the assert
Assert - the actual assertions

Example of an update test:

Arrange: 
    New object as NewObject
    Set properties of NewObject
    Save the NewObject
    Read the object as ReadObject

Act: 
    Change the ReadObject
    Save the ReadObject

Assemble: 
    Read the object as ReadUpdated

Assert: 
    Compare ReadUpdated with ReadObject properties

The reason is so that the ACT does not contain the reading of the ReadUpdated is because it is not part of the act. The act is only changing and saving. So really, ARRANGE ReadUpdated for assertion, I am calling ASSEMBLE for assertion. This is to prevent confusing the ARRANGE section

ASSERT should only contain assertions. That leaves ASSEMBLE between ACT and ASSERT which sets up the assert.

Lastly, if you are failing in the Arrange, your tests are not correct because you should have other tests to prevent/find these trivial bugs. Because for the scenario i present, there should already be other tests which test READ and CREATE. If you create a "Guard Assertion", you may be breaking DRY and creating maintenance.

Answer 6

I don't use that pattern, because I think doing something like:

Arrange
Assert-Not
Act
Assert

May be pointless, because supposedly you know your Arrange part works correctly, which means that whatever is in the Arrange part must be tested aswell or be simple enough to not need tests.

Using your answer's example:

public void testEncompass() throws Exception {
    Range range = new Range(0, 5);
    assertFalse(range.includes(7)); // <-- Pointless and against DRY if there 
                                    // are unit tests for Range(int, int)
    range.encompass(7);
    assertTrue(range.includes(7));
}

Answer 7

Tossing in a "sanity check" assertion to verify state before you perform the action you're testing is an old technique. I usually write them as test scaffolding to prove to myself that the test does what I expect, and remove them later to avoid cluttering tests with test scaffolding. Sometimes, leaving the scaffolding in helps the test serve as narrative.

Answer 8

I've already read about this technique - possibly from you btw - but I do not use it; mostly because I'm used to the triple A form for my unit tests.

Now, I'm getting curious, and have some questions: how do you write your test, do you cause this assertion to fail, following a red-green-red-green-refactor cycle, or do you add it afterwards ?

Do you fail sometimes, perhaps after you refactor the code ? What does this tell you ? Perhaps you could share an example where it helped. Thanks.

Answer 9

Have a look at Wikipedia's entry on Design by Contract. The Arrange-Act-Assert holy trinity is an attempt to encode some of the same concepts and is about proving program correctness. From the article:

The notion of a contract extends down to the method/procedure level; the
contract for each method will normally contain the following pieces of
information:

    Acceptable and unacceptable input values or types, and their meanings
    Return values or types, and their meanings
    Error and exception condition values or types that can occur, and their meanings
    Side effects
    Preconditions
    Postconditions
    Invariants
    (more rarely) Performance guarantees, e.g. for time or space used

There is a tradeoff between the amount of effort spent on setting this up and the value it adds. A-A-A is a useful reminder for the minimum steps required but shouldn't discourage anyone from creating additional steps.

Answer 10

I have done this before when investigating a test that failed.

After considerable head scratching, I determined that the cause was the methods called during "Arrange" were not working correctly. The test failure was misleading. I added a Assert after the arrange. This made the test fail in a place which highlighted the actual problem.

I think there is also a code smell here if the Arrange part of the test is too long and complicated.

Answer 11

In general, I like "Arrange, Act, Assert" very much and use it as my personal standard. The one thing it fails to remind me to do, however, is to dis-arrange what I have arranged when the assertions are done. In most cases, this doesn't cause much annoyance, as most things auto-magically go away via garbage collection, etc. If you have established connections to external resources, however, you will probably want to close those connections when you're done with your assertions or you many have a server or expensive resource out there somewhere holding on to connections or vital resources that it should be able to give away to someone else. This is particularly important if you're one of those developers who does not use TearDown or TestFixtureTearDown to clean up after one or more tests. Of course, "Arrange, Act, Assert" is not responsible for my failure to close what I open; I only mention this "gotcha" because I have not yet found a good "A-word" synonym for "dispose" to recommend! Any suggestions?

Answer 12

Depends on your testing environment/language, but usually if something in the Arrange part fails, an exception is thrown and the test fails displaying it instead of starting the Act part. So no, I usually don't use a second Assert part.

Also, in the case that your Arrange part is quite complex and doesn't always throw an exception, you might perhaps consider wrapping it inside some method and writing an own test for it, so you can be sure it won't fail (without throwing an exception).

Answer 13

If you really want to test everything in the example, try more tests... like:

public void testIncludes7() throws Exception {
    Range range = new Range(0, 5);
    assertFalse(range.includes(7));
}

public void testIncludes5() throws Exception {
    Range range = new Range(0, 5);
    assertTrue(range.includes(5));
}

public void testIncludes0() throws Exception {
    Range range = new Range(0, 5);
    assertTrue(range.includes(0));
}

public void testEncompassInc7() throws Exception {
    Range range = new Range(0, 5);
    range.encompass(7);
    assertTrue(range.includes(7));
}

public void testEncompassInc5() throws Exception {
    Range range = new Range(0, 5);
    range.encompass(7);
    assertTrue(range.includes(5));
}

public void testEncompassInc0() throws Exception {
    Range range = new Range(0, 5);
    range.encompass(7);
    assertTrue(range.includes(0));
}

Because otherwise you are missing so many possibilities for error... eg after encompass, the range only inlcudes 7, etc... There are also tests for length of range (to ensure it didn't also encompass a random value), and another set of tests entirely for trying to encompass 5 in the range... what would we expect - an exception in encompass, or the range to be unaltered?

Anyway, the point is if there are any assumptions in the act that you want to test, put them in their own test, yes?

Answer 14

I use:

1. Setup
2. Act
3. Assert 
4. Teardown

Because a clean setup is very important.