Would elasticsearch or RavenDB be better for fueling a statistics engine/random forest?

Question

I've been looking at the following NoSQL databases for the next phase of my project:

• elasticsearch
• RavenDB

elasticsearch positions itself as primarily serving advanced search scenarios while RavenDB positions itself as a document-oriented-database.

Primarily, the document will be around videos. Each has a natural id. That will be the key of the document.

Around that, I add other content in fields which will not necessarily be scalar or flat, as the information will come from a number of different sources with different structures.

For example, there will be content from the video provider's Atom feeds, blog posts that have the video embedded in it, and other pieces of data from a data warehouse project.

There is no set structure across all of the items (each of them will be very domain-specific, actually), the only thing that will relate them is the natural key of the video mentioned above.

That said, once I have this information in one of the above solutions, I'll want to do a number of things with it:

• Cull it to help populate variables in a random forest in order to make classifications about the videos
• Provide general search on the videos (general free-text, not based on the results of the random forest) through a web-based front end (ASP.NET MVC if you must know)

There are some requirements:

• I will more than likely be in a ASP.NET shared web hosting environment. This means I'll have one machine, and won't have access to set up a service. Something embeddable will be very helpful.

• The ASP.NET environment will be hosted in IIS, so the embeddable aspect will have to survive app-domain recycling.

• I'll want to create new indexes based on the results of the statistical analysis which I can easily fascet which will help with the search on the site.

• Support for autocomplete functionality (I know this isn't an "out-of-the box" request, but being able to get to that point is important).

• Rich synonym support (there's a number of them in the type of videos I'm indexing content around)

I'm also open to services, such as Truffler, although I do have concerns about the cost (and in Truffler's case, a little concerned about latency between the data centers, because the requests will come from the web host on the West coast, or from a back-end process on the East coast).

Additionally, I don't feel that one solution needs to fit all the requirements. I'm more than fine with having one serve one purpose and having another serve another purpose. Granted, migrations suck, but migrating between these two document stores is a little easier (and I don't expect them to use the same document structure, necessarily).

Answer 1

I want to preface this by saying I'm more familiar with Elastic Search so I might be bias. I think RavenDB looks cool, and could probably fit some of your needs well.

Here is why I would vote for Elastic Search.

1. I think your general search, faceting, and synonym support will be easier and more powerful in Elastic Search. Elastic Search leverages so many of the awesome search features from Lucene (i.e. stemming, phonetic, etc.)

2. Elastic Search has better Real Time Searching capabilities. I couldn't exactly figure out if this a strong need of yours, but hey why not have better real time search. Shay explains this very well at Berlin Buzzwords this year.

3. With Elastic Search, you can start with your one server, and scale to many very easily. It was built with the cloud in mind from the start.

There is an Elastic Search .Net API. I'd love to hear what you decided, and how it worked out.