kafka how to pass argument to a consumer using scala

Question

I need to pass argument to my consumer, If i write 1 then my group id is true but i don't know how can pass my parameter. I configured Intelliji to accept parameter but i don't know how can i pass it

Consumer.scala

package utils

import java.util._

import org.apache.kafka.clients.consumer.{ConsumerConfig, KafkaConsumer}

import scala.collection.JavaConverters._

object Consumer {

import java.util.Properties

val TOPIC="topic1"

val  props = new Properties()
props.put("bootstrap.servers", "localhost:9092")
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put(ConsumerConfig.GROUP_ID_CONFIG,"_")


def main(args: Array[String]): Unit = {

if(args(0) == "option==1"){

  props.put(ConsumerConfig.GROUP_ID_CONFIG, "true")

}else if (args(0) == "option==2"){
  props.put(ConsumerConfig.GROUP_ID_CONFIG, false)

}else{
  println("--------")
}

val consumer = new KafkaConsumer[String, String](props)

consumer.subscribe(util.Collections.singletonList(TOPIC))

while (true) {
  val records = consumer.poll(100)
  for (record <- records.asScala) {

    println(record)
    }
  }
}
}

producer.scala

package kafka

import org.apache.log4j.Logger

object KafkaLogger {

val logger = Logger.getLogger(this.getClass.getName)

def main(args: Array[String]): Unit = {

    logger.info("---------Info message---------")

}
}

I launch my producer than my consumer but i don't have a console to pass for example --option=1 or --option=2

OneCricketeer · Answer 1 · 2020-03-29T06:19:51.787

0

args is a Array[String] It'll never hold 1 or 2.

You need to compare == "1" or == "2", or Integer.parseInt(args(0)).

In IntelliJ, you create a run configuration for VM arguments, and your code isn't parsing anything, so you would not type --option. You need a CLI parser for that, or write args(0) == "--option=1"), which would work, I guess

To get around the string problem, use class + def yourMethod(option: Int) rather than object + main(args: Array[String])

Additionally

group id is a string, not a boolean. true/false doesn't really make sense. It seems you are trying to set AUTO_OFFSET_RESET_CONFIG, maybe? Which again, is not a boolean
org.log4j has security flaws and you should use slf4j instead

edited Mar 29 '20 at 06:19

answered Mar 29 '20 at 06:14

OneCricketeer

179,855
19
132
245

thank u , I edited my code but still not able to pass the parameter to the consumer – dor Mar 30 '20 at 07:20
I didn't suggest you to put `option==1`. I told you to use a [CLI Parser](https://stackoverflow.com/questions/2315912/best-way-to-parse-command-line-parameters). See the duplicate post above for the rest of your problem – OneCricketeer Mar 31 '20 at 01:34

kafka how to pass argument to a consumer using scala

1 Answers1