154

Redis can be used as realtime pub-sub just as Kafka.

I am confused which one to use when.

Any use case would be a great help.

nyedidikeke
  • 6,899
  • 7
  • 44
  • 59
Sweta Sharma
  • 2,404
  • 4
  • 21
  • 36

3 Answers3

221

Redis pub-sub is mostly like a fire and forget system where all the messages you produced will be delivered to all the consumers at once and the data is kept nowhere. You have limitation in memory with respect to Redis. Also, the number of producers and consumers can affect the performance in Redis.

Kafka, on the other hand, is a high throughput, distributed log that can be used as a queue. Here any number of users can produce and consumers can consume at any time they want. It also provides persistence for the messages sent through the queue.

Final Take:

Use Redis:

  1. If you want a fire and forget kind of system, where all the messages that you produce are delivered instantly to consumers.
  2. If speed is most concerned.
  3. If you can live up with data loss.
  4. If you don't want your system to hold the message that has been sent.
  5. The amount of data that is gonna be dealt with is not huge.

Use kafka:

  1. If you want reliability.
  2. If you want your system to have a copy of messages that has been sent even after consumption.
  3. If you can't live up with data loss.
  4. If Speed is not a big concern.
  5. data size is huge
sky
  • 260
  • 3
  • 12
Karthikeyan Gopall
  • 5,469
  • 2
  • 19
  • 35
  • 116
    One main difference is that Redis Pub/Sub is push based while Kafka Pub/Sub is pull based. That means messages published to Redis will be automatically delivered to subscribers instantly, while in Kafka Data/messages are never pushed out to consumers, the consumer will ask for messages when the consumer is ready to handle the message. https://www.cloudkarafka.com/blog/2016-11-30-part1-kafka-for-beginners-what-is-apache-kafka.html http://kafka.apache.org/documentation.html#design_pull – Zeni Apr 03 '18 at 06:09
38

Redis 5.0+ version provides the Stream data structure. It could be considered as a log data structure with delivery guarantees. It offers a set of blocking operations allowing consumers to wait for new data added to a stream by producers, and in addition to that, a concept called Consumer Groups.

Basically Stream structure provides the same capabilities as Kafka.

Here is the documentation https://redis.io/topics/streams-intro

There are two most popular Java clients that support this feature: Redisson and Jedis

Redisson provides ReliableTopic object if reliability of delivery is required. https://github.com/redisson/redisson/wiki/6.-distributed-objects/#613-reliable-topic

Nikita Koksharov
  • 10,283
  • 1
  • 62
  • 71
0

I think the most diffrent between these two service is

  1. Redis supports push-based delivery of messages that means messages published to Redis will be delivered automatically to subscribers immediately but kafka is supports pull-based delivery of messages, meaning that messages published in Kafka are never distributed directly to consumers, consumers subscribe to topics and ask for messages when consumers are ready to deal with them.

  2. Redis does not support the concept of parallelism but Kafka supports parallelism due to the log partitioning of data where multiple consumers consume in consumer groups at the same.

  3. Redis sends the messages to the consumer all at once and later the message is removed. Thus, no one knows where the data is held but kafka is a log, there are always messages; you can monitor this by setting a retention policy for messages. E.g. 7 days retention

  4. redis is used for various use cases such as Session Cache, Full Page Cache (FPC), Leader Boards/Counting, Pub s/ Sub, Queues but kafka has various use cases such as Messaging, Website Activity Tracking, Log Aggregation, Stream Processing, Metrics, Event Sourcing, Commit log.

Behdin Talebi
  • 1