How to assert the number of assertions (or some other way to enforce that all members have been tested in an assertion)?

Question

In a certain unit test for my class, I want to assert that an assertion has been written for all member fields. That is, given

public class Foo {
  int a;
  int b;
  String c;
}

I want to do something like

@Test
public void testFieldsAgainstExternalModel(Foo model, Bar externalModel) {
  assertEquals(model.a, externalModel.getA());
  assertEquals(model.b, externalModel.getSomethingNamedDifferently());
  assertEquals(model.c, externalModel.getC().toString());
  assertNumberOfAssertionsInThisTest(3); // <-- How do I do this?
}

Of course, counting the number of assertions doesn't ensure that they had anything to do with each field having an assertion. But, I'm just trying to write a test that fails if a developer adds a field to Foo and forgets to update this test.

• Naivest approach: The simplest thing I can think of is to use reflection to assert the number of fields Foo has, e.g. assertEquals(3, Foo.getDeclaredFields().count()). But, I can easily see a programmer bumping up the number without actually adding an assertion.

• Still a naive approach, but acceptable: I think if I can at least count the number of assertions, it would sufficiently guide programmers toward the intended need.

• More exact approaches: Of course it seems to best approach is to actual keep a table of which fields have appeared in the assertion (e.g. through a wrapper method like assertEqualsWithCounters), but in reality my situation's a little more complicated. For example, my model is actually a generated class (@AutoMatter) and I'll be using getters on both sides of the assertion, so it'll be hard to infer truly which field has been "touched" by an assertion. That's why just counting assertions and remaining agnostic to actual types and counts, would be easiest for me.

Answer 1

But, I'm just trying to write a test that fails if a developer adds a field to Foo and forgets to update this test.

My company uses a mapping framework to map beans from the db, fields are one to one with the database, to a model bean we return to the client that we can modify as needed. I believe this is a common use case and essentially what you are trying to get at. You should be less concerned with testing changes to Foo, and more concerned with ensuring nothing breaks with the mapping from Foo to Bar.

Class BeanDb{
 private int a;
 private int b;
 ...
}

Class Bean{
 private int a;
 private int b;
 private String somethingElse;
 ...
}

Then we have have a proivder test that uses isEqualToComparingFieldByFieldRecursively to compare the two beans.

private Bean bean = new Bean();
private Bean beanDb = new BeanDb();
private BeanProvider provider = new BeanProvider();

@Before
public void setup(){
 bean.setA(1);
 bean.setB(2);

 beanDb.setA(1);
 beanDb.setB(2);
}

@Test
public void prepareForDatabase(){
    BeanDb db = provider.prepareForDatabase(bean);

    assertThat(beanDb).isEqualToComparingFieldByFieldRecursively(db);
}

@Test
public void populateFromDatabase(){
    Bean model = provider.populateFromDatabase(beanDb);

    assertThat(model).isEqualToComparingFieldByFieldRecursively(bean);
}

This actually catches a lot of bugs. If something changes in the model bean / db bean the test will break in our continuous integrations suite. The caveat here is that isEqualToComparingFieldByFieldRecursively will not catch newly added fields. When I review code / pull requests and BeanDb or the provider has been updated I ask, "Did you update the provider test?"

Answer 2

I think you may need to use PowerMockito as Mockito does not support mocking static methods.

I haven't tested the code below - but my idea would be to do the following:

@RunWith(PowerMockRunner.class)
public class Tester {
    public class Foo {
        int a = 5;
        int b = 6;
        String c = "7";
    }
    class Bar {

        public int getA() {
            return 5;
        }

        public int getSomethingNamedDifferently() {
            return 6;
        }

        public Integer getC() {
            return 7;
        }
    }

    @Test
    public void testFieldsAgainstExternalModel() {
        testFieldsAgainstExternalModel(new Foo(), new Bar());
    }
    public void testFieldsAgainstExternalModel(Foo model, Bar externalModel) {
        Assert spiedAssert = PowerMockito.spy(Assert.class);

        spiedAssert.assertEquals(model.a, externalModel.getA());
        spiedAssert.assertEquals(model.b, externalModel.getSomethingNamedDifferently());
        spiedAssert.assertEquals(model.c, externalModel.getC().toString());
        PowerMockito.verify(spiedAssert, times(3)).assertEquals(any(), any());
    }
}

Answer 3

I think that the answer to

assertNumberOfAssertionsInThisTest(3); // <-- How do I do this?

is: you do not do that. This test is close to useless.

If there are only 3 fields, you can see with your eyes that you are checking those three fields. You are probably thinking about 15 fields in that class, but then:

assertEquals(model.a, externalModel.getA());
assertEquals(model.b, externalModel.getSomethingNamedDifferently());
assertEquals(model.c, externalModel.getC().toString());
assertEquals(model.c, externalModel.getC().toString());
assertEquals(model.d, externalModel.getD().toString());

... assertEquals(model.o, externalModel.getO().toString());

assertNumberOfAssertionsInThisTest(15); 

would not help you. Guess what: when pulling together my faked example, I omitted property 14 (model.n), but instead I had a copy/paste error and checked model.c two times, leading to an overall count of 15.

Therefore I agree with user Phil, just suggesting yet another approach, like this:

• ensure that the bean classes have reasonable equals() (and maybe hashCode() methods). Either by using Lombok, or by having a base bean class that makes use of an EqualsBuilder (we actually do the later in our projects)
• then have test code either use a faked bean and simply compare if that faked bean is equal to the bean that your production code created from a (maybe faked) external model object.