3

We have some applications which want to communicate with Kafka using REST API calls to both consume and produce messages. If we do not want to use Confluent REST Proxy, what are the options ?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Anirban
  • 257
  • 4
  • 12
  • 1
    Your question needs more context. REST Proxy is a good solution, so what's the reason not to use it? That will drive the answer. If it's technical, what technical issue? If it's something else, what is that reason? – Robin Moffatt Jan 23 '20 at 22:52
  • Hi Robin, the reason is monetary. – Anirban Jan 23 '20 at 23:26
  • 1
    [REST Proxy is completely open source](https://github.com/confluentinc/kafka-rest). There is no monetary attachment to it – OneCricketeer Jan 23 '20 at 23:28
  • Hi @cricket_007, We are getting Confluent Kafka and the configuration we received didn't take into account Rest Proxy. So we want to know if there is any alternative solution. – Anirban Jan 23 '20 at 23:34
  • @Anirban There is no such thing as Confluent Kafka. You can add REST Proxy to any existing Apache Kafka environment. – OneCricketeer Jan 23 '20 at 23:34
  • To put it another way, if the "monetary" reason is actually "extra hardware", then yeah, you'll need that anyway with any other alternative – OneCricketeer Jan 23 '20 at 23:35
  • Hi @cricket_007, We are working with Confluent team. This requirement is kind of new. So we want to know what additional hardware we need for that. Alternatively we want to know if there is consumer/producer that we can create to meet the requirement which can save us money. – Anirban Jan 23 '20 at 23:37
  • If you are working with sales/engineering, then they should steer you in the right direction. Why are you trying to find alternatives to something that is already free? And does not cost anything to **try**? – OneCricketeer Jan 23 '20 at 23:39
  • Surely Confluent has sent you here as well https://docs.confluent.io/current/kafka-rest/deployment.html#production-deployment – OneCricketeer Jan 23 '20 at 23:40
  • The real alternative that would match your specific business use cases, is to write your own API, which embeds a standard consumer/producer internally. – OneCricketeer Jan 23 '20 at 23:43
  • 1
    Hi @cricket_007, We are thinking of not buying any additional hardware for this new request and use existing configuration to meet the requirement.I am mostly interested to know if consumer/producer can be created to meet this requirement. – Anirban Jan 23 '20 at 23:46
  • Okay, then pick an existing server with at least 2GB available of memory, and run `kafka-rest-start` and see how well it works? Sorry, I don't know what you mean by "consumer/producer can be created" becuase you're not clear on your requirements other than using REST – OneCricketeer Jan 23 '20 at 23:47
  • 1
    Hi @cricket_007, I will check kafka-rest-start. I was thinking if we can create Rest-API calls which will be used by other applications to consume data from Kafka and push data to Kafka – Anirban Jan 23 '20 at 23:54
  • *we can create Rest-API calls which will be used by other applications to consume data from Kafka and push data to Kafka* -- Yes, that is exactly the point of the REST Proxy – OneCricketeer Jan 24 '20 at 17:45

2 Answers2

4

One possible alternative is the Strimzi Kafka Bridge (https://github.com/strimzi/strimzi-kafka-bridge). It's part of the broader Strimzi project about running Kafka on Kubernetes but work even running as standalone (when your Kafka cluster is on bare metal). Of course it's open source and Apache 2.0 licensed.

ppatierno
  • 9,431
  • 1
  • 30
  • 45
1

the reason [not to use it] is monetary

You can use the Confluent REST Proxy with no software/licensing costs.

We are thinking of not buying any additional hardware for this new request and use existing configuration to meet the requirement.I am mostly interested to know if consumer/producer can be created to meet this requirement

You don't need extra hardware.

Pick an existing server with at least 2GB available of memory, and run kafka-rest-start and see how well it works

if we can create Rest-API calls which will be used by other applications to consume data from Kafka and push data to Kafka

That's the main purpose of REST Proxy, yes.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245