1

I am just starting with Kafka, it sounds really good for Microservices, but I work essentially in Scala.

I added kafka to my sbt project with this:

libraryDependencies += "org.apache.kafka" %% "kafka" % "2.0.0"

Then I do this:

import org.apache.kafka.clients.producer.{Callback,KafkaProducer, Producer}

...

val producer = new KafkaProducer[String, String](props)
val record = new ProducerRecord[String, String]("my-topic", "key", "value")
val fut = producer.send(record, callBack)
...

My problem here is that I am not getting a Scala Future when I call producer.send, it is a Java Future. I don't know how Java Futures work, and I would prefer to skip that learning curve. This time it is Future, but I mean Java in general.

So I am wondering if there is a full Scala api to work with Kafka. It should normally be the case since Kafka is written in Scala.

piet.t
  • 11,718
  • 21
  • 43
  • 52
acmoune
  • 2,981
  • 3
  • 24
  • 41
  • 1
    skip java's learning curve while programming in scala is a bit ironic : ). Take a look here https://stackoverflow.com/questions/35033290/convert-a-java-future-to-a-scala-future – aran May 17 '19 at 06:36
  • @aran: Scala is a simpler language than Java. It has fewer corner cases and special cases. Then, there is Java's Type System, which according to Martin Odersky is only understood by 3 people, … and he should know since he designed large portions of it! (In fact, from the tone of the email, I got the impression that he does not count himself among those 3 people, despite the fact that he designed large portions of Java's Type System.) For comparison: the Scala Language Specification is less than 150 pages including a Changelog and some of the core libraries. The JLS is 750 pages w/o libs. – Jörg W Mittag May 17 '19 at 07:08
  • @aran, I know using java packages (so java) is unavoidable. I was just saying that since I am coding in Scala anyway, I hope there is a Scala api so I will not have to spend all time integrating Java code into Scala code, like in your link. I think my concern make sense, but based on the feedback I imagine that there is no Scala api. Thanks anyway. – acmoune May 17 '19 at 10:09
  • the reason of my comment is the deep (as you say, unavoidable) union between java and scala. For example, I believe scala's concurrent package is, in fact, a tag for java's package. But I see your question as valid, so rebalanced the votes. – aran May 17 '19 at 10:13
  • Ok. The point is not just that I don't know java future, it is that I prefer scala future. It would have been cool to have a version of `producer.send` that returns it. – acmoune May 17 '19 at 10:22
  • Finally I will give a try to RabbitMQ, there is good support and examples for Scala/Akka, that is the only reason. – acmoune May 17 '19 at 15:16
  • Did you look into Kafka streams? There is a Scala DSL for it! – joesan May 20 '19 at 15:23
  • Yes I know, it should cover everything that really has to be done. – acmoune May 20 '19 at 21:05

1 Answers1

1

From Kafka notable changes in 2.0.0

  • The Scala consumers, which have been deprecated since 0.11.0.0, have been removed. The Java consumer has been the recommended option since 0.10.0.0. Note that the Scala consumers in 1.1.0 (and older) will continue to work even if the brokers are upgraded to 2.0.0.

  • The Scala producers, which have been deprecated since 0.10.0.0, have been removed. The Java producer has been the recommended option since 0.9.0.0. Note that the behaviour of the default partitioner in the Java producer differs from the default partitioner in the Scala producers. Users migrating should consider configuring a custom partitioner that retains the previous behaviour. Note that the Scala producers in 1.1.0 (and older) will continue to work even if the brokers are upgraded to 2.0.0.

Gery
  • 609
  • 4
  • 9