6

I'm looking for a simple DBM library such as BerkeleyDB or Kyoto Cabinet, but natively working for .NET. I need to manage more than one million records (maybe ten millions), and possibly in a hierarchical way (this last point is not mandatory, we can work with a linear structure).

Does such a thing even exist?

picrap
  • 1,236
  • 1
  • 13
  • 35

5 Answers5

5

I finally found RaptorDB and it matches almost exactly my needs. I'll write down my evaluation here after it's been fully tested.

Edit: after testing, it was not exactly working as expected. Actually it was not exactly working, so I wrote my own DBM (it always ends up like this :)). The problems where in a multithreaded environnement, where I got DB corruption quite often (and yes, I've checked my code for thread safety). If someone has a positive feedback, I'm interested.

picrap
  • 1,236
  • 1
  • 13
  • 35
  • I'm also experimenting with RaptorDB, but my requirements are much less than yours. I simply need a unique key value store to persist json geocoding results for several months. Essentially a non-volatile cache of RESTful service responses. RaptorDB seems like a good fit! – NovaJoe Mar 13 '12 at 15:51
4

SQL Server Compact Edition is a viable choice, although I'm not sure what you mean by "natively working for .NET" - .NET has no built-in database (unless you count DataTables and XML, which I don't).

MusiGenesis
  • 74,184
  • 40
  • 190
  • 334
  • 1
    I'd prefer something written in .NET, because the application is going to be multi-platform. I'd also rather like a simple library. – picrap Oct 12 '11 at 19:46
  • I know of no database engines written in .NET (and it would be kind of crazy to try). SqlCE is a very simple library to use, and it could handle a million records with no sweat. – MusiGenesis Oct 12 '11 at 19:49
  • 2
    Why look for a "simple library" when you can get the (almost) full functionality of SQL Server? For free? – John Saunders Oct 12 '11 at 19:50
  • 2
    because typical tables such as BerkeyDB or Kyoto Cabinet are much more faster :) (and beside, I just need a simple key/value storage) – picrap Oct 12 '11 at 20:03
  • @picrap: that's a very risky and unfounded statement of yours... can you back it up with hard facts and numbers?? I highly doubt that .... – marc_s Oct 12 '11 at 20:29
  • 1
    http://en.wikipedia.org/wiki/Dbm but otherwise, I couldn't find any benchmark between a RDBM and a DBM. There is no speed benchmark between a car and a bike neither, but the result is usually obvious too :) – picrap Oct 13 '11 at 06:02
1

What about System.Data.SQLite?

Vlad
  • 18,195
  • 4
  • 41
  • 71
  • 1
    I'm not sure about SQLite performance (I updated my question, because I need to be able to search a million records fast). I'm also not sure for file defragmentation to work. – picrap Oct 12 '11 at 19:42
1

Assuming you are on Windows I suggest trying the ESENT database engine via the ManagedEsent interop layer. ESENT is built into Windows and has a similar level of functionality and performance to BerkelyDB. The differences are that ESENT isn't just a key/value store, it has columns and indexes built in, but ESENT doesn't have replication.

Laurion Burchall
  • 2,843
  • 16
  • 12
  • Thanks, but the application is going to be multi-platform, so native code, and especially system components, is not the best option. – picrap Oct 12 '11 at 19:52
0

My company has built a .NET layer for the Berkeley DB XML library, which you can download at http://bdbxml.net. You get an embedded XML database library that you can run in .NET and can use the databases on other platforms using the different language bindings if need be. If you're interested, feel free to contact me with further requirements so I can understand your solution architecture a little better and see if we can be a fit for you.

Regards

Lucas Vogel
  • 51
  • 2