0

We have a data system in which writes and reads can be made in a couple of geographic locations which have high network latency between them (crossing a few continents, but not this slow). We can live with 'last write wins' conflict resolution, especially since edits can't be meaningfully merged.

I'd ideally like to use a distributed system that allows fast, local reads and writes, and copes with the replication and write propagation over the slow connection in the background. Do the datacenter-aware features in e.g. Voldemort or Cassandra deliver this?

It's either this, or we roll our own, probably based on collecting writes using something like rsync and sorting out the conflict resolution ourselves.

Community
  • 1
  • 1
Jim Downing
  • 1,481
  • 12
  • 29
  • If you're going to downvote, it would be great if you could add a comment to let me know why? I thought this a valid SO question. – Jim Downing Jul 19 '12 at 05:32
  • Apologies and thanks to both chris and blahdiblah. Apparently although both are great answers, I'm not allowed to split the bounty (see http://meta.stackexchange.com/questions/2786/accept-multiple-answers-or-split-bounty-among-several-users). – Jim Downing Jul 30 '12 at 07:47

2 Answers2

0

You should be able to get the behavior you're looking for using Voldemort. (I can't speak to Cassandra, but imagine that it's similarly possible using it.)

The key settings in the configuration will be:

  • replication-factor — This is the total number of times the data is stored. Each put or delete operation must eventually hit this many nodes. A replication factor of n means it can be possible to tolerate up to n - 1 node failures without data loss.

  • required-reads — The least number of reads that can succeed without throwing an exception.

  • required-writes — The least number of writes that can succeed without the client getting back an exception.

So for your situation, the replication would be set to whatever number made sense for your redundancy requirements, while both required-reads and required-writes would be set to 1. Reads and writes would return quickly, with a concomitant risk of stale or lost data, and the data would only be replicated to the other nodes afterwards.

Community
  • 1
  • 1
blahdiblah
  • 33,069
  • 21
  • 98
  • 152
0

I have no experience with Voldemort, so I can only comment on Cassandra.

You can deploy Cassandra to multiple datacenters with an inter-DC latency higher than a few milliseconds (see http://spyced.blogspot.com/2010/04/cassandra-fact-vs-fiction.html).

To ensure fast local reads, you can configure the cluster to replicate your data to a certain number of nodes in each datacenter (see "Network Topology Strategy"). For example, you specify that there should always be two replica in each data center. So even when you lose a node in a data center, you will still be able to read your data locally.

Write requests can be sent to any node in a Cassandra cluster. So for fast writes, your clients would always speak to a local node. The node receiving the request (the "coordinator") will replicate the data to other nodes (in other datacenters) in the background. If nodes are down, the write request will still succeed and the coordinator will replicate the data to the failed nodes at a later time ("hinted handoff").

Conflict resolution is based on a client-supplied timestamp.

If you need more than eventual consistency, Cassandra offers several consistency options (including datacenter-aware options).

chris
  • 217
  • 3
  • 1