Tests using RavenDb EmbeddableDocumentStore very slow compared to real local Raven instance

Question

In summary: I want to unit test classes that use an IDocumentSession using an EmbeddableDocumentStore (the recommended approach I believe). Compared to using a session created with a "real", locally-hosted DocumentStore - a simple test using a static index runs very slowly. Having run some basic profiling - about 7x slower (see below). I can't use RavenTestBase as utimately I want to use this approach to test using SpecsFor.

I was expecting tests using the in-memory store to be very fast - am I doing something wrong to explain this performance?

EDIT: Having read this I disabled my anti-virus, this has made no difference.

The difference between the two approaches:

I create an EmbeddableDocumentStore by:

    private static IDocumentStore CreateInMemoryEmbdeddableDocumentStore()
    {
        var embeddedStore = new EmbeddableDocumentStore();
        embeddedStore.Configuration.RunInMemory = true;
        embeddedStore.Configuration.RunInUnreliableYetFastModeThatIsNotSuitableForProduction = true;
        embeddedStore.RegisterListener(new NoStaleQueriesAllowedListener());
        return embeddedStore.Initialize();
    }

and the "real" DocumentStore by:

    public IDocumentStore CreateLocalDocumentStore()
    {
        var store = new DocumentStore { Url = "http://localhost:8080", DefaultDatabase = "ColourTest" };
        store.RegisterListener(new NoStaleQueriesAllowedListener());
        return store.Initialize();
    }

In both cases I am using this to ensure indexing has taken place before getting query results:

    public class NoStaleQueriesAllowedListener : IDocumentQueryListener
    {
        public void BeforeQueryExecuted(IDocumentQueryCustomization queryCustomization)
        {
            queryCustomization.WaitForNonStaleResults();
        }
    }

I then get a session by:

    private static IDocumentSession ConfigureIndexesAndCreateSession(IDocumentStore store)
    {
        IndexCreation.CreateIndexes(typeof(Colours_ColourCountByMood).Assembly, store);
        return store.OpenSession();
    }

Using this session, I setup test data (with the real store, I also delete existing data first). As usual I then run the method being tested - which uses a static index - and do an assert.

Comparing performance with profiling:

I also added logging calls through the code (see complete code here) to compare timing in both cases. Sample results of a test run:

TestUsingInMemoryEmbeddable()
  191 ms : Start create EmbeddableDocumentStore
 1819 ms : Finish create EmbeddableDocumentStore
 1819 ms : Start embeddedStore.Initialize()
 3411 ms : Finish embeddedStore.Initialize()
 3411 ms : Start CreateIndexes
 5322 ms : Finish CreateIndexes
 5322 ms : Start OpenSession
 5330 ms : Finish OpenSession
 5331 ms : Start test data storing
 5852 ms : Finish test data storing
 5853 ms : Start test Act
 6985 ms : Finish test Act
 6985 ms : Start test Assert
 6998 ms : Finish test Assert

TestUsingLocallyHosted()
    1 ms : Start create DocumentStore
    1 ms : Finish create DocumentStore
    2 ms : Start documentStore.Initialize()
  608 ms : Finish documentStore.Initialize()
  608 ms : Start CreateIndexes
  717 ms : Finish CreateIndexes
  717 ms : Start OpenSession
  717 ms : Finish OpenSession
  718 ms : Start DeleteTestData
  730 ms : Finish DeleteTestData
  730 ms : Start test data storing
  823 ms : Finish test data storing
  823 ms : Start test Act
  957 ms : Finish test Act
  957 ms : Start test Assert
  957 ms : Finish test Assert

Having run these a number of times the above figures are representative: when running embedded, the relative slowness is apparent/inherent throughout the test code.

score 0 · Answer 1 · answered Mar 23 '14 at 08:05

0

You need to use the RunInMemory property directly on the EmbeddableDocumentStore.

Also, you might want to run that several times, to see the timing. We are doing a lot of upfront work to reduce overall costs.

answered Mar 23 '14 at 08:05

Ayende Rahien

22,925
1
36
41

I've run this again a few times but get similar results. [Taking a look at the source](https://github.com/ravendb/ravendb/blob/master/Raven.Client.Embedded/EmbeddableDocumentStore.cs), it looks like calling "embeddedStore.Configuration.RunInMemory = true;" and "embeddedStore.RunInMemory = true;" do the same thing - RunInMemory just sets the corresponding property on its RavenConfiguration instance. – rogersillito Mar 24 '14 at 11:10
If I duplicate (copy, paste and then rename) [my test method "TestUsingInMemoryEmbeddable()"](https://gist.github.com/rogersillito/9692046#file-raventestembeddedvsserverwithprofiling-cs) multiple times within my test fixture, the first copy that runs performs as above, but all subsequent copies run very quick (200ms). This suggests that even though within each copy I am creating a store/session **and then disposing each of these** - _something_ hangs around in memory so that the in-memory store gets created and initialised much quicker on subsequent test invocations within the same run... – rogersillito Mar 26 '14 at 12:38
If so then the performance issue I thought would be multiplied for each time I create/initialise an EmbeddableDocumentStore (i.e. each class I test where I need an IDocumentSession) is much less of a problem. Does this fit with your understanding? – rogersillito Mar 26 '14 at 12:39

Tests using RavenDb EmbeddableDocumentStore very slow compared to real local Raven instance

1 Answers1