Redis streams - remove / expire events based on time

Question

I am playing with redis stream and it is good so far. I am trying to understand if there is anyway for me to expire the old events based on time or some other way. I know that we can remove by event id. But I do not want to remember / store the event id which is difficult. Instead I am looking for a way remove the last 10K events or something like that.

Answer 1

This is possible as of Redis 6.2.

If you use the default event IDs (by passing * as an ID to XADD) they will begin with the UNIX timestamp of when the event was inserted, followed by a dash.

Then you can use XTRIM $stream_name MINID $timestamp to remove all events with an ID lower than '$timestamp', which is equivalent to all events older than the timestamp.

Answer 2

So far, there's no way to expire events by time. Instead, the only expire strategy is to expire events by keeping the latest N events. You can use the XTRIM command to evict old events.

Should i do that very time? Can stream be configured to retain the last N events ?

If you want to always keep the latest N events, you can call XADD command with MAXLEN option to get a capped stream. Also with ~ option, you can have better performance, but inaccurately expire events. Check the doc for detail.

UPDATE

Since Redis 6.2, XTRIM supports a new trimming strategy: MINID. With this strategy, Redis will evict entries whose ids are lower than the given threshold.

So if you use timestamp as entry id, e.g. the default, auto-generated id use Unix timestamp (in milliseconds) as part of the id, you can use this strategy to expire events based on time, i.e. remove events older than the given timestamp.