-1

i have a distributed application. here are set of processes , spread accross mutiple computers , communicating each other. i have a data structure , which is modified among these proceses . and this is not stored in database .

Now the question is how do i maintain the same view of the this data structure , accross all processes

i.e., at any point of time all process should see the same data structure

Meghasyam
  • 1,021
  • 2
  • 9
  • 7
  • 1
    Sounds like a difficult problem; I suggest reading http://www.powells.com/biblio?isbn=9780201633382 for detailed information on protocols to manage stale/live data. Sure, it's intended for Unix kernel authors (talk about a small audience), but I think the protocols described may be useful to you too. – sarnold Jan 24 '11 at 08:56
  • Does your model have a master, then make all the slaves use the master's data structure which gets updated the instant some change occurs. Or each slave has the same copy as the master, and whenever the master changes, the slaves are also updated. – DumbCoder Jan 24 '11 at 09:00
  • That really depends on the data structure you're using, I think. Can you provide more details about what you're doing? – templatetypedef Jan 24 '11 at 09:34
  • it is a key value pair data structure – Meghasyam Jan 24 '11 at 11:12

3 Answers3

1

You say that you don't have a database. That's a shame, because database authors have solved your problem. You would need to incorporate the equivalent technology in your project. And obviously, the fastest and most simple way to incorporate the technology of databases is to incorporate a database.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • 2
    hi please dont answer like this . this is not all i wanted for my question . if you know the answer then do it otherwise plz dont not this is very shame on you . please keep quiet – Meghasyam Jan 24 '11 at 09:30
  • 2
    Look, i *DO* know the answer. You want a database. I can't help that you don't like the answer. But if you want a bit of software that synchronizes datastructures across computers and processes, we in the industry call that a database. If you can't deal with standard terminology, please don't ask questions on SO. – MSalters Jan 24 '11 at 09:55
1

Redis is designed to solve your problem. It is a key-value store for sharing between programs running on different machines but sharing the data. It is a server you run somewhere, and your programs all connect to this server using the client library it provides.

You can also use a database such as mysql but with in-memory tables.

If your data-structure does not fit into the key-value or relational models very well, you have the same kind of situation as multi-player games. It is non-trivial to sync multi-player games but it can be done and here is an excellent introduction as to how: gafferongames.com

Will
  • 73,905
  • 40
  • 169
  • 246
  • Hi , my data strucutre fit into key-value pair . so can you please elobrate on this that how this can be done – Meghasyam Jan 24 '11 at 11:08
0

I would recommend something like the Data Distribution Services platform for something like this (open source version is OpenDDS). Their key selling point is that it is designed to propagate changes to data to all interested in such changes. And performance isn't bad either.

Commercial implementations of this protocol are used in a variety of real-time systems, mostly military grade applications.

More options to consider, distributed caches (such as memcached) - though I've not played with this myself - it looks quite straight forward to get up and running.

Nim
  • 33,299
  • 2
  • 62
  • 101