0

According to the schema data comes to Kafka, then to stream and Mapr-DB.

enter image description here

After storing data in DB, user can display data on the map.

Question is, why we use DB to dispaly data on the map if Kafka is already DB.

It seems to me more slowly to get realtime data from Mapr-DB that from Kafka.

What do you think, why this example uses this appoarch?

Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156

1 Answers1

1

The core abstraction Kafka provides for a stream of records is known as topic. You can imagine topics as the tables in a database. A database (Kafka) can have multiple tables (topics). Like in databases, a topic can have any kind of records depending on the usecase. But note that Kafka is not a database.

Also note that in most cases, you would have to configure a retention policy. This means that messages at some point will be deleted based on a configurable time or size based retention policy. Therefore, you need to store the data into a persistent storage system and in this case, this is your Database.

You can read more about how Kafka works in this blog post.

Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
  • Thank you, I got you. In case when I store all data from Kafka to DB it has delay in realtime? Why we talk about real time if Kafka msut store data in DB then anyone consumer should read data from DB in realtime? Is it possible witj low latancy? Could you anser me, If I need to display data in realtime, should I before store this in DB? –  May 17 '20 at 12:10
  • @AliceMessis Kafka is able to process high volume of messages assuming you configure both the cluster and your pipeline in a way that meets your requirements. – Giorgos Myrianthous May 17 '20 at 12:18
  • You have a retention policy but you can set it to be permanent... Please see this topic: https://stackoverflow.com/a/39735773/11521880 . – bridgemnc Dec 05 '22 at 21:32