How do I plug custom partition logic in WriteKafkaP?

Question

I'm using the WriteKafaP vertex in a Jet job. Is there a way to plug in custom partition logic when publishing to the Kafka topic?

You mean you want to determine the partition, to which the event will be published? — Oliv, Nov 09 '17 at 15:13

score 2 · Accepted Answer · answered Nov 09 '17 at 16:19

In 0.4, you can use a custom Kafka partitioner configuration. In 0.5, we added the ability to map the item to a ProducerRecord directly.

For example:

Vertex v = new Vertex("kafkaSink", KafkaProcessors.writeKafkaP(properties, o -> new ProducerRecord(topic, partition, key, value));

1 Answers1