17

I'm evaluating the use of Azure Event Hub vs Kafka as a Service Broker. I was hoping I would be able to create two local apps side by side, one that consumes messages using Kafka with the other one using Azure Event Hub. I've got a docker container set up which is a Kafka instance and I'm in the process of setting up Azure Event hub using my Azure account (as far as I know there's no other way to create a local/development instance for Azure Event Hub).

Does anyone have any information regarding the two that might be useful when comparing their features?

Nore Gabbidon
  • 351
  • 1
  • 4
  • 10
  • I always thought they are pretty much the same thing but from different brands and platforms. it comes down to whether you want to manage events with Kafka or Azure. if I understand correctly - i could be wrong, just guessing here - you can run Kafka on azure as well. – Jay Sep 18 '20 at 10:18

5 Answers5

11

Can't add a comment directly, but the currently top rate answer has the line

Kafka can have multiple topics each Azure Event Hub is a single topic.

This is misleading as it makes it sound like you can't have multiple topics, which you can. As per https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview#kafka-and-event-hub-conceptual-mapping an "Event Hub" is a topic while an "Event Hub Namespace" is the Kafka cluster.

Euan
  • 150
  • 2
  • 6
9

This decision usually is driven by a broader architectural choice if you are choosing azure as your iaas and paas solution then event hub provides a great integration within the azure ecosystem but if you don't want a vendor lock in kafka is better option.

Operationally also if you want fully managed service then with event hub it's out of the box but with kafka you also get this with confluent platform.

Maturity wise kafka is older and with large community you have a larger support.

Feature wise what kafka ecosystem provides azure ecosystem has those things but if you talk about only event hub then it lacks few features compared to kafka

I think this link can help you extend your understanding https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview

Yatin Goyal
  • 322
  • 3
  • 10
6

Kafka can have multiple topics each Azure Event Hub is a single topic. Kafka running inside a container means you have to manage it. Azure Event Hub is a PaaS which means they managed the platform side. If you don't know how to make Kafka redundant, reliable, and scalable you may want to go with Azure Event Hubs or any PaaS that offers a similar pub/sub model. Event Hub platform is already scalable, reliable, and redundant.

Steve Fibich
  • 314
  • 1
  • 2
  • 7
  • 1
    A Kafka cluster can have multiple topics. Similarly, an Event Hub Namespace can have multiple Event Hubs. Azure Event Hub Namespace is a logical container that can hold multiple Even Hub instances. So, both Kafka and Event Hub are similar on this aspect! – Azhar Ansari Jun 04 '22 at 17:27
6

While Apache Kafka is software you typically need to install and operate, Event Hubs is a fully managed, cloud-native service. There are no servers, disks, or networks to manage and monitor and no brokers to consider or configure, ever. You create a namespace, which is an endpoint with a fully qualified domain name, and then you create Event Hubs (topics) within that namespace. For more information about Event Hubs and namespaces, see Event Hubs features. As a cloud service, Event Hubs uses a single stable virtual IP address as the endpoint, so clients don't need to know about the brokers or machines within a cluster. Even though Event Hubs implements the same protocol, this difference means that all Kafka traffic for all partitions is predictably routed through this one endpoint rather than requiring firewall access for all brokers of a cluster. Scale in Event Hubs is controlled by how many throughput units you purchase, with each throughput unit entitling you to 1 Megabyte per second, or 1000 events per second of ingress and twice that volume in egress. Event Hubs can automatically scale up throughput units when you reach the throughput limit if you use the Auto-Inflate feature; this feature work also works with the Apache Kafka protocol support.

You can find more on feature comparison here - https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview

Serkant Karaca
  • 1,896
  • 9
  • 8
2

You should compare

  • the administration capabilites / effort (as previously said)
  • the functional capabilities such as competing customer and pub/sub patterns
  • the performance : you should consider kafka if you plan to exceed the event hub quotas
Crazywolf
  • 56
  • 2