Embedded PostgreSQL for Java JUnit tests

Question

Is there an embedded PostgreSql so that we could unit test our PostgreSql driven application?

Since PostgreSql has some dialects, it's better to use embedded PostgreSql itself than other embedded databases.

Embedded does not necessarily mean it must be embedded in the JVM process. It also does not necessarily need to use in-memory persistence. It should be loaded automatically by the dependency management (Maven, Gradle), so that Unit tests can run on every machine without having to install and configure a local PostgreSQL server.

Answer 1

There is an "embedded" PostgresSQL server that has been designed for unit testing from Java:

https://github.com/yandex-qatools/postgresql-embedded

Embedded postgresql will provide a platform neutral way for running postgres binary in unit tests. Much of the code has been crafted from Flapdoodle OSS's embed process

As an aside, there also exists similar projects for Mongo, Redis, Memcached and nodejs.

Answer 2

No, there is no embedded PostgreSQL, in the sense of an in-process-loadable database-as-a-library. PostgreSQL is process oriented; each backend has one thread, and it spawns multiple processes to do work. It doesn' make sense as a library.

The H2 database supports a limited subset of the PostgreSQL SQL dialect and the use of the PgJDBC driver.

What you can do is initdb a new temporary database, start it with pg_ctl on a randomized port so it doesn't conflict with other instances, run your tests, then use pg_ctl to stop it and finally delete the temporary database.

I strongly recommend that you run the temporary postgres on a non-default port so you don't risk colliding with any locally installed PostgreSQL on the machine running the tests.

(There is "embedded PostgreSQL in the sense of ecpg, essentially a PostgreSQL client embedded in C source code as preprocessor based C language extensions. It still requires a running server and it's a bit nasty to use, not really recommended. It mostly exists to make porting from various other databases easier.)

Answer 3

I tried the project suggested by @btiernay (yandex-qatools). I spent a good few days with this and without any offence it's over engineered solution which doesn't work in my case as I wanted to download the binaries from internal repository rather than going to public internet. In theory it supports it but in fact it doesn't.

OpenTable Embedded PostgreSQL Component

I ended up using otj-pg-embedded and it works like a charm. It was mentioned in comments so I thought I'll mention it here as well.

I used it as standalone DB and not via rule for both unit tests and local development.

Dependency:

<dependency>
    <groupId>com.opentable.components</groupId>
    <artifactId>otj-pg-embedded</artifactId>
    <version>0.7.1</version>
</dependency>

Code:

@Bean
public DataSource dataSource(PgBinaryResolver pgBinaryResolver) throws IOException {
    EmbeddedPostgres pg = EmbeddedPostgres.builder()
        .setPgBinaryResolver(pgBinaryResolver)
        .start();


    // It doesn't not matter which databse it will be after all. We just use the default.
    return pg.getPostgresDatabase();
}

@Bean
public PgBinaryResolver nexusPgBinaryResolver() {
    return (system, machineHardware) -> {
        String url = getArtifactUrl(postgrePackage, system + SEPARATOR + machineHardware);
        log.info("Will download embedded Postgre package from: {}", url);

        return new URL(url).openConnection().getInputStream();
    };
}

private static String getArtifactUrl(PostgrePackage postgrePackage, String classifier) {
    // Your internal repo URL logic
}

Answer 4

You can use a container instance of PostgreSQL.

Since spinning a container is a matter of seconds, this should be good enough for unittests. Moreover, in case you need to persist the data, e.g. for investigation, you don't need to save the entire container, only the data files, which can be mapped outside of the container.

One of example of how to do this can be found here.

Answer 5

I am using the container instance of PostgreSQL in the tests. https://www.testcontainers.org/#about https://www.testcontainers.org/modules/databases/jdbc/

dependencies:

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.7.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.7.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.7.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers</artifactId>
            <version>1.15.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>1.15.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>postgresql</artifactId>
            <version>1.15.3</version>
            <scope>test</scope>
        </dependency>

And do the tests:

@SpringBootTest
@ActiveProfiles({"test"})
@Testcontainers
class ApplicationTest {
    @Container
    static PostgreSQLContainer<?> postgreSQL = new PostgreSQLContainer<>("postgres:12.7");

    @DynamicPropertySource
    static void postgreSQLProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.username", postgreSQL::getUsername);
        registry.add("spring.datasource.password", postgreSQL::getPassword);
    }


    @Test
    void someTests() {

    }

in application-test.yml:

source:
  datasource:
    url: jdbc:tc:postgresql:12.7:///databasename

Answer 6

If you are looking to run an in-process version of postgres from an Integration (or similar) test suite, the postgresql-embedded worked fine for me.

I wrote a small maven plugin that can be used as a maven wrapper around a forked version of postgresql-embedded.