34

we have the following use case: Every time a certain key expires, we need to get notified and do something, based on it's value. But when redis fires the expired event, the key was already removed from the db when we try to access it later on, which is expected of course.

Now is there a way to access the entry again, after it expired? I guess not.

So second option: Is there a way to tell redis to publish the whole value object instead of just the key when sending those events? I guess it could be added through Lua, but I'd interested in an easier option, if possible. We also need this behaviour for other events, we basically need all notifications to publish the value, not the key (we could do a GET once the event was received, but we want to get around the second call, primarily to have an atomic process, since the value could have changed between publishing the event and doing the GET to retrieve the value).

Hope it's understandable. Maybe we can't see the obvious, so thanks in advance!

Amin Shojaei
  • 5,451
  • 2
  • 38
  • 46
Daniel Torres
  • 749
  • 1
  • 7
  • 17
  • Redis 2.8 is still a release candidate so far. If you need control about key expiration notifications, it would probably be better to implement it explicitly (without any dependency on 2.8). See http://stackoverflow.com/questions/11810020/how-to-handle-session-expire-basing-redis/11815594#11815594 – Didier Spezia Aug 20 '13 at 08:54
  • 2
    Well that 2.8 is still RC isn't the problem. The problem is that Redis publishes events with the key, instead of the value of the entry. And since we need this behaviour for all notification, also 2.6. ones, we need a way to tell Redis to give us the whole object at the time the event was triggered, instead of the key. – Daniel Torres Aug 20 '13 at 09:14
  • 1
    I would still use a solution based on zset – Didier Spezia Aug 20 '13 at 14:56
  • 1
    i've created an issue on github as i also need this feature https://github.com/antirez/redis/issues/1876 – Eugene Platonov Jul 16 '14 at 14:00

2 Answers2

68

The feature that Eli linked to allows you to listen when a key expires. However, it does not give you the value of the key. Futhermore, based on the filed github issue it does not look like you can expect to have this feature built in anytime soon if ever. The solution I use is to create a special "shadow" expiration key that is linked to the key where you have an actual value.

So lets say you have a key called testkey and it has an integer value of 100. Furthermore, the key will expire after 10 seconds at which point you want to get the value of the key. (Maybe you were incrementing the key during the 10 seconds it existed).

First you need to setup listening for keyspace events. In particular you want to listen for expired events. You can do this from your config or use the config set command in redis. (see here for more info: http://redis.io/topics/notifications)

CONFIG SET notify-keyspace-events Ex

Now you can subscribe to a special keyevent channel where you will be notified that the key expired.

SUBSCRIBE __keyevent@0__:expired

The format of the channel to subscribe to is __keyevent@<db>__:<eventName>. In our example we're assuming we're working with the default database 0 and we want to listen for the expired event.

When the testkey expires you will now get a message in the __keyevent__ channel where the message is the name of the key that expired. Of course at this point the key is gone so we can no longer access the value! The solution is to use a special expiration key.

When you create your testkey also create a special expiring "shadow" key (don't expire the actual testkey). For example:

SET testkey 100
SET shadowkey:testkey "" EX 10

Now in the __keyevent@0__:expired channel you will get a message telling you that the key shadowkey:testkey expired. Take the value of the message (which is the name of the key), split on the colon (or whatever separator you decide to use), and then manually get the value of the key and delete it.

// set your key value
SET testkey 100 
//set your "shadow" key, note the value here is irrelevant
SET shadowkey:testkey "" EX 10 
// Get an expiration message in the channel __keyevent@0__:expired
// Split the key on ":", take the second part to get your original key
// Then get the value and do whatever with it
GET testkey
// Then delete the key
DEL testkey

Note that the value of the shadowkey isn't used so you want to use the smallest possible value which according to this answer (Redis store key without a value) is an empty string "". It's a little more work to setup but the above system does exactly what you need. The overhead is a few extra commands to actually retrieve and delete your key plus the storage cost of an empty key.

Motti
  • 110,860
  • 49
  • 189
  • 262
Sarus
  • 3,303
  • 1
  • 23
  • 27
  • Nice idea although it does double that value's footprint. A possible tweak is only storing the value in the "shadow" key and using the actual key just for expiration purposes. – Itamar Haber Feb 22 '15 at 10:51
  • 1
    @ItamarHaber The shadowkey doesn't store the actual value. In fact, based on this answer (http://stackoverflow.com/questions/25557250/redis-store-key-without-value) the shadow key value should just be an empty string (in my example I gave it a value of "1" but have now updated it). So the only overhead is that of an extra key (you should not be duplicating your value) – Sarus Feb 23 '15 at 00:56
  • 1
    Really nice strategy! – leonsPAPA Apr 28 '17 at 17:25
  • 1
    There is a caveat though. If your client breaks before getting the event or deleting the key that contains the actual value, you will end up contaminating your redis with stale keys. A solution to this might be to set the actual key to also expire, maybe 10 secs after the shadow key so that you'll have time to get it after the shadow key expires and then ignore events on non-shadow keys. – axxis Jan 17 '21 at 11:45
  • 1
    Is there a way to receive only expired events for a specific key? – gemr1423 Apr 01 '22 at 21:58
1

If you're on 2.8, you can try out this new feature (also referenced at this page). It's definitely unstable and doesn't seem well tested, but if you're on 2.8 anyway...

Short intro from the issue page:

An interesting feature in databases with a key-value data model (Redis does not perfectly fit this definition as the values are complex data structure, but the outer layer of Redis is definitely a key-value business) is the ability to subscribe in some way to the stream of events targeting a given key.

For instance I may be interested to see when the key foo is deleted or modified in some way, or to get the names of all the keys with an expire set (using the EXPIRE command) that Redis is evicting from the dataset because their time to live dropped to zero.

This feature was requested many times by the Redis user base, however so far we never reached a point where the proposed API (including proposals made by myself) seemed to fit well into the Redis design. This feature request will try to describe a new design that is extremely simple to use, to implement, and that fits well in the Redis story.

Eli
  • 36,793
  • 40
  • 144
  • 207