7

I'm using memcahced (specifically the Enyim memcached client) and I would like to able to make a keys in the cache dependant on other keys, i.e. if Key A is dependent on Key B, then whenever Key B is deleted or changed, Key A is also invalidated.

If possible I would also like to make sure that data integrity is maintained in the case of a node in the cluster fails, i.e. if Key B is at some point unavailable, Key A should still be invalid if Key B should become invalid.

Based on this post I believe that this is possible, but I'm struggling to understand the algorithm enough to convince myself how / why this works.

Can anyone help me out?

Community
  • 1
  • 1
Justin
  • 84,773
  • 49
  • 224
  • 367

2 Answers2

7

I've been using memcached quite a bit lately and I'm sure what you're trying to do with depencies isn't possible with memcached "as is" but would need to be handled from client side. Also that the data replication should happen server side and not from the client, these are 2 different domains. (With memcached at least, seeing its lack of data storage logic. The point of memcached though is just that, extreme minimalism for bettter performance)

For the data replication (protection against a physical failing cluster node) you should check out membased http://www.couchbase.org/get/couchbase/current instead.

For the deps algorithm, I could see something like this in a client: For any given key there is a suspected additional key holding the list/array of dependant keys.

# - delete a key, recursive:
function deleteKey( keyname ):
    deps = client.getDeps( keyname ) #
    foreach ( deps as dep ):
        deleteKey( dep )
        memcached.delete( dep )
    endeach
    memcached.delete( keyname )
endfunction

# return the list of keynames or an empty list if the key doesnt exist
function client.getDeps( keyname ):
    return memcached.get( key_name + "_deps" ) or array()
endfunction

# Key "demokey1" and its counterpart "demokey1_deps". In the list of keys stored in
# "demokey1_deps" there is "demokey2" and "demokey3".
deleteKey( "demokey1" );
# this would first perform a memcached get on "demokey1_deps" then with the
# value returned as a list of keys ("demokey2" and "demokey3") run deleteKey()
# on each of them.

Cheers

smassey
  • 5,875
  • 24
  • 37
0

I don't think it's a direct solution but try creating a system of namespaces in your memcache keys, e.g. http://www.cakemail.com/namespacing-in-memcached/. In short, the keys are generated and contain the current values of other memcached keys. In the namespacing problem the idea is to invalidate a whole range of keys who are within a certain namespace. This is achieved by something like incrementing the value of the namespace key, and any keys referencing the previous namespace value will not match when the key is regenerated.

Your problem looks a little different, but I think that by setting up Key A to be in the Key B "namespace, if a node B was unavailable then calculating Key A's full namespaced key e.g. 

"Key A|Key B:<whatever Key B value is>"

will return false, thus allowing you to determine that B is unavailable and invalidate the cache lookup for Key A.

Ian
  • 977
  • 8
  • 14