13

I've been working on a project for over half a year now, building healthcare software from the ground up. When I joined up, MySQL had been chosen as the primary data store.

A few months and many headaches later, we've begun to investigate alternative data stores that can offer the flexibility we need to record our critical and ever-changing healthcare data.

We've looked at many NoSQL solutions; MongoDB drawing the most of our attention. Being able to store structured, embedded data would be a huge benefit. We've been scared off by reports of data loss/reliability issues, however.

I've come across a few "NewSQL" data stores and I'm interested in VoltDB in particular.

I'm curious to know if anyone has any experience with Volt or has seen it implemented in a project.

Edit:

Data integrity and consistency are most important. It could be very harmful for a patients information to be lost, they may receive improper treatment etc.

Data volume will vary; we will likely support small practices first. Something like 700 users total. But even when we scale up to hospitals, we're not looking at social media like traffic.

Regarding your question, yes data structures will evolve. On top of having to change the existing structure to capture new or modified inputs, we have to preserve the structure of the existing data as a sort of snap-shot. We've only been able to do this EAV style with MySQL.

Thanks for your feedback.

jthurau
  • 447
  • 1
  • 3
  • 10
  • well, you know, MySQL is the least reliable SQL database.. after maybe SQLite.. And even Oracle databases blow up. So.. –  Feb 14 '12 at 22:47
  • Investigating Mongo as an alternative led me to VoltDB and was thinking maybe those in a similar situation might find a discussion involving the two to be useful – jthurau Feb 14 '12 at 23:50
  • This article highlights that you don't get high availability if you upgrade the schema or change the scale (no elasticity). Also no concurrency for long queries like reports or bulk changes. http://pgsnake.blogspot.co.nz/2010/05/comparing-voltdb-to-postgres.html?m=1 – KCD Apr 10 '14 at 18:57

3 Answers3

35

We went live last year with an application that uses VoltDB. We're storing around 1.5 billion records and processing 50-90 million transactions a day with a kfactor=1 4 server cluster ( 256 GB memory/server ). Given the performance of VoltDB, we could easily be handing 1 billion transactions a day.

To date, we have had no problems related to the VoltDB software. Our experience is that it is truly ACID compliant. With the addition of the Command Logging feature, I believe you can configure the logging parameters to preclude the loss of any transactions.

Other strong features include its scalability ( and the relative simplicity to add capacity ).

An important consideration when choosing VoltDB is understanding VoltDB's partitioning scheme. Achieving the extremely high transaction rates possible with VoltDB depends on the parallelism achieved through data partitioning. The partitioning is transparent to your application, but your application data must lend itself to being partitioned to get the maximum performance. If your data does not lend itself to partitioning, I believe the primary impact would be reduced throughput ( i.e. transaction rates ) - not a show-stopper.

Finally - a note concerning stored procedures. VoltDB allows you to replace stored procedures without stopping the database. Also, each invocation of a stored procedure constitutes a single transaction. We have leveraged stored procedures in such a way that we are able to modify/update the our application logic without stopping the database.

StevieE
  • 366
  • 2
  • 2
  • 1
    StevieE - is it possible to speak with you regarding your VoltDB experience in more details? – Daniil Aug 08 '12 at 13:32
0

The question is a little bit old, but I give some feedback because we've been using MongoDB since a long time, and we are looking to VoltDB but for totally different reasons:

  • Regarding mongoDB: we are using mongoDB in production since 4 years and we never lost any single byte of data. The "mongodb is not reliable" seems to be a myth, at least for me. We are managining millions of new entries in mongoDB every days without any problems

  • We are looking to VoltDB for a different use case: providing real time analytics on huge volume. MongoDB is not bad at providing analytics, but when you go beyond the millions of entries to be analyzed, it start to be a little bit slow. VoltDB is much better at this, but I would not advice you to use it to store data, especially high value data.... We use another database to store the data.

tomsoft
  • 4,448
  • 5
  • 28
  • 35
  • VoltDB should be able to handle millions for sure. But Please note that voltDB's selling point is not big data, but fast data. That's what they do really well. It depends on the requirements. But for big data voltDB -may- not be the best solution available. –  Dec 24 '16 at 01:32
  • I wonder how far you can push it with VoltDB. There are several OLAP-(like) solutions that will smartly aggregate and index the data. Linkedin's Pinot looks interesting... VoltDB's awesomeness solves alot of problems. But providing metrics from massive datasets may not be one of them :-) –  Dec 24 '16 at 01:44
  • I agree that it is not a bad idea to combine a database like mongoDB from a persistence point of view. But in my opinion VoltDB is better at correctly processing writes under high concurrency (by design). Technically, but also the way developers work with both databases. VoltDB separates transactions and runs them (effiently and- ) much closer to the data than is the case in Mongo apps. Mistakes can be made in both cases. I feel like mongo applications often have more responsibilies when it comes to handling concurrency etc and therefore I consider them more error prone to human errors. –  Dec 24 '16 at 02:25
0

The question, as it stands, is very subjective, but more information can help us point you in the right direction.

What exactly are your requirements? Does this system have transactional needs where data integrity and consistency are of the utmost importance, such as those in trading and financial applications? What is the volume of data, and how many concurrent users? What are the performance requirements?

Also, you mentioned ever-changing healthcare data, does this mean the data structures are constantly changing and evolving? If so, you may want to stay away from relational databases given the nature of rigid schemas and, instead, consider document stores such as Mongo where schemaless data structures provides more flexibility.

BTW,

Don't get scared about reliability stories on Mongo; you can find horror stories for practically any product, both open source and commercial; often times these bad reviews have got less to do with the product and more to do with poor customer implementation.

VoltDB, last I checked, requires that all persistence logic be coded as stored procedures. The obvious shortcomings with this approach is less code visibility and modularity, and higher maintenance needs. Aside from that, Voltdb is a very fast since most of the overhead found in traditional relational databases, such as locking, etc, are eliminated from the core database engine.

raffian
  • 31,267
  • 26
  • 103
  • 174
  • Please explain what you mean by stored procedures causing higher maintenance needs. – Kuberchaun Mar 14 '12 at 00:23
  • 1
    In my experience, SP's cause maintenance headaches because they interact with tables, and the RDBMS in general, directly; change the table and the SP will likely change as well. I prefer using a data services/abstraction layer approach when interacting with data; it's not a good idea to implement persistence logic specific to the data store type. Here's a quick story: we used Sybase for 10 years, implemented over 1000 stored procs in it, but when Sybase changed their licensing structure, we had to migrate to Oracle. The stored proc conversion alone took 2 1/2 years and cost us millions. – raffian Mar 14 '12 at 17:25