How to populate data in testcontainers?

Question

CONTEXT:

I have a non Spring Boot app which I wrap in Spring Boot in order to test functionality. It happened that code had an in-memory database in tests, I want to test against a different database to reproduce the production environment. I decided to go with testcontainers to keep all existing tests untouched.

PROBLEM:

I need to load millions of rows into the test container. The question is how to populate the database when in testcontainers? I found a similar question -> How to populate testcontainers? but I still cannot get how to populate data in it.

How can I populate data in testcontainers?

DatabaseTestInitalizer.java I use for instantiating container:

import org.springframework.boot.test.util.EnvironmentTestUtils;
import org.springframework.context.ApplicationContextInitializer;
import org.springframework.context.ConfigurableApplicationContext;
import org.testcontainers.containers.MSSQLServerContainer;

public class DatabaseTestInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {

    private MSSQLServerContainer mssqlServerContainer;

    @Override
    public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
        mssqlServerContainer = new MSSQLServerContainer();
        mssqlServerContainer.start();

        // This is solution for 1.x.x Spring Boot framework
        // Article for migration from 1.x.x to 2.x.x Spring Boot https://stackoverflow.com/questions/54718995/appropriate-usage-of-testpropertyvalues-in-spring-boot-tests
        EnvironmentTestUtils.addEnvironment(configurableApplicationContext.getEnvironment(),
                "spring.datasource.url=" + mssqlServerContainer.getJdbcUrl(),
                "spring.datasource.username=" + mssqlServerContainer.getUsername(),
                "spring.datasource.password=" + mssqlServerContainer.getPassword()
        );
    }
}

You should populate it manually in the "Before" block in your test class. Or you can use `docker-entrypoint-initdb.d` mechanism. Check here "Initializing a fresh instance" section: https://hub.docker.com/_/mysql — Héctor, Jun 18 '20 at 08:47
do you use Flyway to populate your database schema or do you use an MySQL Docker image which already cotnains your schema? — rieckpil, Jun 19 '20 at 06:30
@rieckpil No, the reason why I don't really need it. In production, I just read from the database. I am trying to create a test to reproduce production conditions in a test. Flyway, Liquibase, or other tools will be overkill and unnecessary complication for the project. I think I found a solution `.withInitScript` but it is just slow. — Dmytro Chasovskyi, Jun 19 '20 at 11:47

score 7 · Accepted Answer · answered Jun 25 '20 at 13:02

As it came out after a more rigorous search I can use withInitScript and define the schema plus data into init.sql, make sure that db/init.sql resides into test/resources folder.

It works as below:

public MSSQLServerContainer mssqlServerContainer = (MSSQLServerContainer) new MSSQLServerContainer()
            .withInitScript("db/init.sql")
            .withExposedPorts(1433);

In this case, you don't need Flyway.

score 1 · Answer 2 · answered Jun 18 '20 at 11:13

1

you have two options:

create container from dockerfile Links : testcontainer dockerfile Customize your MySQL Database in Docker
you can use a database migration tools like flyway and database will automatically populated

answered Jun 18 '20 at 11:13

El Merrouni

77
4

score 1 · Answer 3 · answered Jun 18 '20 at 12:13

When I worked with a similar setup I've implemented the following approach:

I've managed the schema in Flyway which is integrated with spring boot anyway. All the migrations were Schema definition related migrations and did not do anything to the data. I've placed them in src/main/sesources/... (not the test classpath) - because they were also relevant for the regular application startup.
I've created special migrations for tests that populate the test data. These migrations were under src/test/resources of my module. These "test" migrations were written in a way that they've always started after the regular migrations.
I've configured Flyway to start before the test so that it has populated the test data.

After all, the main idea is to decouple the code of test from the data generation.

It worked for us - so I can recommend this approach.

Having said that, if you're using spring, there are other ways:

Add the data "manually" in a setup if you're using JPA. Place the call to this code before tests kick in.
Use @Sql annotation to run a script with test data. All-in-all there are many options there, read this tutorial for more information

How to populate data in testcontainers?

3 Answers3