1

I’m building a system for playing cards, and I’ve decided on a pub/sub system using SSE to keep the clients updated. A problem I’m facing is keeping messages around for later querying. One use case is a player joins a table mid game. I’d like to fetch all the events from the current hand and replay them for that user.

I’m currently using redis’ pub/sub, but the messages are entirely ephemeral. I’ve done some digging into rabbitMQ and it also seems to be the case. Kafka seems to have some replay functionality? I can’t tell if it’s as granular as I’d like, though.

Is it reasonable to expect persistence from my pub/sub layer? Or should I separate the logic? I could have a consumer running just to take the events and store them. That just seems heavy, though, with too much opportunity for failure.

I’d also like to query the old events for stats on the players, and the cards played. Speed isn’t a factor on those queries, though.

Dudo
  • 4,002
  • 8
  • 32
  • 57

2 Answers2

1

I don't know any pub/sub message distributor which supports persistentance for messages.

Running a consumer which will save the messages in a persistent storage is a valid solution. But I think that for your use case it might be worth to use streams instead. It's supported by both Kafka and redis.

ofirule
  • 4,233
  • 2
  • 26
  • 40
1

Take a look at Redis Streams, it is part of Redis as of v5.

What are the main differences between Redis Pub/Sub and Redis Stream?

It will persist your messages. Using consumer groups, consumers can be at different offsets of the stream

https://redis.io/topics/streams-intro

LeoMurillo
  • 6,048
  • 1
  • 19
  • 34
  • I read up a bit. Don't the consumer groups ensure that a message is only delivered to 1 subscriber? – Dudo May 11 '20 at 19:21