Embedded MongoDB when running integration tests

Question

My question is a variation of this one.

Since my Java Web-app project requires a lot of read filters/queries and interfaces with tools like GridFS, I'm struggling to think of a sensible way to employ MongoDB in the way the above solution suggests.

Therefore, I'm considering running an embedded instance of MongoDB alongside my integration tests. I'd like it to start up automatically (either for each test or the whole suite), flush the database for every test, and shut down at the end. These tests might be run on development machines as well as the CI server, so my solution will also need to be portable.

Can anyone with more knowledge on MongoDB help me get idea of the feasibility of this approach, and/or perhaps suggest any reading material that might help me get started?

I'm also open to other suggestions people might have on how I could approach this problem...

Answer 1

I have found Embedded MongoDB library which looks quite promising and does what you have asked for.

Currently supports MongoDB versions: 1.6.5 to 3.1.6, provided the binaries are still available from the configured mirror.

Here is short example of use, which I have just tried and it works perfectly:

public class EmbeddedMongoTest {
    private static final String DATABASE_NAME = "embedded";

    private MongodExecutable mongodExe;
    private MongodProcess mongod;
    private Mongo mongo;

    @Before
    public void beforeEach() throws Exception {
        MongoDBRuntime runtime = MongoDBRuntime.getDefaultInstance();
        mongodExe = runtime.prepare(new MongodConfig(Version.V2_3_0, 12345, Network.localhostIsIPv6()));
        mongod = mongodExe.start();
        mongo = new Mongo("localhost", 12345);
    }

    @After
    public void afterEach() throws Exception {
        if (this.mongod != null) {
            this.mongod.stop();
            this.mongodExe.stop();
        }
    }

    @Test
    public void shouldCreateNewObjectInEmbeddedMongoDb() {
        // given
        DB db = mongo.getDB(DATABASE_NAME);
        DBCollection col = db.createCollection("testCollection", new BasicDBObject());

        // when
        col.save(new BasicDBObject("testDoc", new Date()));

        // then
        assertThat(col.getCount(), Matchers.is(1L));
    }
}

Answer 2

Here's an updated (for 2022) version of the accepted answer from @rozky (a lot has been changed in both the Mongo and Embedded MongoDB libraries).

package com.example.mongo;

import com.mongodb.BasicDBObject;
import com.mongodb.MongoClient;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import de.flapdoodle.embed.mongo.MongodExecutable;
import de.flapdoodle.embed.mongo.MongodProcess;
import de.flapdoodle.embed.mongo.MongodStarter;
import de.flapdoodle.embed.mongo.config.MongodConfig;
import de.flapdoodle.embed.mongo.config.Net;
import de.flapdoodle.embed.mongo.distribution.Version;
import de.flapdoodle.embed.process.runtime.Network;
import java.util.Date;
import org.junit.After;
import static org.junit.Assert.*;
import org.junit.Before;
import org.junit.Test;

public class EmbeddedMongoTest {

    private static final String DATABASE_NAME = "embedded";

    private MongodExecutable mongodExe;
    private MongodProcess mongod;
    private MongoClient mongo;

    @Before
    public void beforeEach() throws Exception {
        MongodStarter starter = MongodStarter.getDefaultInstance();
        String bindIp = "localhost";
        int port = 12345;
        MongodConfig mongodConfig = MongodConfig.builder()
            .version(Version.Main.PRODUCTION)
            .net(new Net(bindIp, port, Network.localhostIsIPv6()))
            .build();
        this.mongodExe = starter.prepare(mongodConfig);
        this.mongod = mongodExe.start();
        this.mongo = new MongoClient(bindIp, port);
    }

    @After
    public void afterEach() throws Exception {
        if (this.mongod != null) {
            this.mongod.stop();
            this.mongodExe.stop();
        }
    }

    @Test
    public void shouldCreateNewObjectInEmbeddedMongoDb() {
        // given
        MongoDatabase db = mongo.getDatabase(DATABASE_NAME);
        db.createCollection("testCollection");
        MongoCollection<BasicDBObject> col = db.getCollection("testCollection", BasicDBObject.class);

        // when
        col.insertOne(new BasicDBObject("testDoc", new Date()));

        // then
        assertEquals(1L, col.countDocuments());
    }

}

Answer 3

There is Foursquare product Fongo. Fongo is an in-memory java implementation of mongo. It intercepts calls to the standard mongo-java-driver for finds, updates, inserts, removes and other methods. The primary use is for lightweight unit testing where you don't want to spin up a mongo process.

Answer 4

If you're using Maven you may be interested in a plugin I've created that wraps the flapdoodle.de 'embedded mongo' API:

embedmongo-maven-plugin

It provides a start goal that you can use to start any version of MongoDB you want (e.g. during pre-integration-test), and a stop goal that will stop MongoDB (e.g. during post-integration-test).

The real benefit of using this plugin over others is that there is no requirement for MongoDB to be installed beforehand. MongoDB binaries are downloaded and stored in ~/.embedmongo for future builds.

Answer 5

If you are using sbt and specs2, I wrote the same kind of wrapper for embedmongo

https://github.com/athieriot/specs2-embedmongo

Answer 6

with spring-boot 1.3 you can use EmbeddedMongoAutoConfiguration

pom.xml

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.3.2.RELEASE</version>
</parent>
 ...
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-mongodb</artifactId>
    </dependency>
    <dependency>
        <groupId>de.flapdoodle.embed</groupId>
        <artifactId>de.flapdoodle.embed.mongo</artifactId>
        <version>${embedded-mongo.version}</version>
    </dependency>

MongoConfig

@Configuration
@EnableAutoConfiguration(exclude = { EmbeddedMongoAutoConfiguration.class })
public class MongoConfig{
}

Answer 7

You can run MongoDB in memory as of version 3.2.6. From the site:

Starting in MongoDB Enterprise version 3.2.6, the in-memory storage engine is part of general availability (GA) in the 64-bit builds. Other than some metadata and diagnostic data, the in-memory storage engine does not maintain any on-disk data, including configuration data, indexes, user credentials, etc.

Answer 8

Not just for unit testing, but also explained how to use inmemory mongodb with rest api.

maven dependency:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>

        <dependency>
            <groupId>de.flapdoodle.embed</groupId>
            <artifactId>de.flapdoodle.embed.mongo</artifactId>
        </dependency>

=============================================================================

application.properties

server.port = 8080
spring.data.mongodb.database=user_db
spring.data.mongodb.port=27017
spring.data.mongodb.host=localhost

=============================================================================

UserRepository.java

public interface UserRepository extends MongoRepository{

}

for reference and all java code use below link:(step by step explanation)

https://www.youtube.com/watch?v=2Tq2Q7EzhSA&t=7s

Answer 9

Performances are better when executing mongod with storageEngine='ephemeralForTest'

new MongodConfigBuilder()
    .version(Version.Main.PRODUCTION)
    .cmdOptions(new MongoCmdOptionsBuilder()
         .useStorageEngine("ephemeralForTest")
         .build())
    .net(new Net("localhost", port, Network.localhostIsIPv6()))
    .build()

Answer 10

To run Embedded mongodb for integration test following are the maven dependency needed:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
            <version>2.5.2</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
            <version>2.5.2</version>
        </dependency>

        <dependency>
            <groupId>de.flapdoodle.embed</groupId>
            <artifactId>de.flapdoodle.embed.mongo</artifactId>
            <version>3.0.0</version>
            <scope>test</scope>
        </dependency>

Try using below code snipped for EmbeddedMongoAutoConfiguration:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.mongo.MongoProperties;
import org.springframework.boot.autoconfigure.mongo.embedded.EmbeddedMongoAutoConfiguration;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
public class EmbeddedMongoApplication {

    public static void main(String[] args) {
         System.setProperty("os.arch", "x86_64");
         SpringApplication.run(EmbeddedMongoApplication.class, args);
    }
    
    @Bean
    public EmbeddedMongoAutoConfiguration embeddedMongoAutoConfiguration(MongoProperties mongoProperties) {
        return new EmbeddedMongoAutoConfiguration(mongoProperties);
    }
}

Note:

Embedded mongodb will be downloaded to below Path. So take into consideration that path have proper permission.

Linux : $HOME/.embedmongo/linux/mongodb-linux-x86_64-3.2.2.tgz
Windows : C:\Users\<username>\.embedmongo\win32\mongodb-win32-x86_64-3.x.x.zip

Answer 11

In production, you will be using a real database.

If you want your tests to reflect how your product behaves in production, use a real instance of Mongo.

A fake implementation may not behave exactly the same as a real one. When testing, you should strive for correctness. Speed of execution comes second.