Junit strange behavior that cause @ParameterizedTest to fail

Question

I have a dead simple test case with the most basic set up:

@Entity
public class Tags {

  @Id
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  private Integer id;

  @Column(unique = true)
  private String name;

  public Tags(String name) {
    this.name = name;
  }
}

@Repository
public interface TagsRepository extends JpaRepository<Tags, Integer> {}

Test set up:

@DataJpaTest
public class TagsTest implements WithAssertions {

  @Autowired
  private TagsRepository tagsRepo;

  @BeforeEach
  void setUp() {
    Tags tag1 = new Tags("tag1");
    Tags tag2 = new Tags("tag2");
    Tags tag3 = new Tags("tag3");

    tagsRepo.saveAll(List.of(tag1, tag2, tag3));
  }

  @AfterEach
  void cleanUp() {
    tagsRepo.deleteAll();
  }

  @Test
  void test() {
    Tags actual1 = tagsRepo.findById(1).orElse(null);
    assertThat(actual1).isNotNull();  // <- passed
  }

  @Test
  void test2() {
    Tags actual2 = tagsRepo.findById(1).orElse(null);
    assertThat(actual2).isNotNull();  // <- failed
  }
  
  @Test
  void test3() {
    Tags actual3 = tagsRepo.findById(2).orElse(null);
    assertThat(actual3).isNotNull();  // <- failed
  }
}

Unexpected behavior:

When run test class as a whole, only the first test passed and second and third test all failed.

And this leads to problem in @ParameterizedTest as well:

@ParameterizedTest
@ValueSource(ints = { 1, 1, 2 }) // <- second and third test failed
void test4(Integer id) {
  Tags actual = tagsRepo.findById(id).orElse(null);
  assertThat(actual).isNotNull();
}

Same behavior after changing dataSource to @MethodSource or @CsvSource.

I tried to initialize data in each test, but same thing happend:

@Test
void test() {
  Tags tag1 = new Tags("tag1");
  Tags tag2 = new Tags("tag2");
  Tags tag3 = new Tags("tag3");
  
  tagsRepo.saveAll(List.of(tag1, tag2, tag3));
  
  Tags actual1 = tagsRepo.findById(1).orElse(null);
  
  assertThat(actual1).isNotNull();
  
  tagsRepo.deleteAll();
}

Why would only the first test passed and subsequent test failed? I've tried using different combinations of id passed in. But all the same behavior.

With this simple demo, I came to the conclusion that, running similar test will casue issue, even though, test should not influence each other?

I know I shouldn't test the built-in API provided by SpringBoot out of the box, but this is a issue I discovered when testing mybatis with mutiple params using @Parameterized.

Edit: Seems like most folks tried to solve this by re-creating database before each test

Answer 1

It doesn't work because the counter for the ID goes on and on every time you are deleting/saving the data in your setUp() method:

@BeforeEach
  void setUp() {
    Tags tag1 = new Tags("tag1");
    Tags tag2 = new Tags("tag2");
    Tags tag3 = new Tags("tag3");

    tagsRepo.saveAll(List.of(tag1, tag2, tag3));
  }

for the first test you will have:

tag1-> id = 1
tag2-> id = 2
tag3-> id = 3

but for the second one you will have:

tag1-> id = 4
tag2-> id = 5
tag3-> id = 6

and so on... so the right way would be to save each of them and keep in a class level variable, so something like this:

@DataJpaTest
public class TagsTest implements WithAssertions {

    @Autowired
    private TagsRepository tagsRepo;

    private Tags tag1;
    private Tags tag2;
    private Tags tag3;
    
    @BeforeEach
    void setUp() {
        tag1 = tagsRepo.save(new Tags("tag1"));
        tag2 = tagsRepo.save(new Tags("tag2"));
        tag3 = tagsRepo.save(new Tags("tag3"));
    }

    @AfterEach
    void cleanUp() {
        tagsRepo.deleteAll();
    }

    @Test
    void test() {
        Tags actual1 = tagsRepo.findById(tag1.getId()).orElse(null);
        assertThat(actual1).isNotNull();
    }

    @Test
    void test2() {
        Tags actual2 = tagsRepo.findById(tag1.getId()).orElse(null);
        assertThat(actual2).isNotNull();
    }

    @Test
    void test3() {
        Tags actual3 = tagsRepo.findById(tag2.getId()).orElse(null);
        assertThat(actual3).isNotNull();
    }
}

Regarding the parametrized test, you cannot really know the id in advance using an auto incremented id