0

We are using Aiven as a Kafka as a Service provider, and they support credential rotation. E.g. every 24 hours.

So far, we have yet to find a way to create new consumers / producers in Spring Kafka when we need to rotate. Basically, a TopicAuthorizationException is thrown on consumer.poll.

What we need to do, is basically create a new consumer with a update truststore and keystore path. Keystore and truststore is loaded from kubernetes secrets.

Anyone know how we can make this work?

Lorna Mitchell
  • 1,819
  • 13
  • 22
Kenneth Gunnerud
  • 208
  • 4
  • 13

1 Answers1

1

The listener container will stop after any AuthorizationException.

You can add an ApplicationListener or @EventListener method to receive a ListenerContainerStoppedEvent.

You can either wait for all child containers to stop, or immediately stop the concurrent container; update the container's containerProperties.kafkaConsumerProperties with the new store locations and restart the container.

I don't currently see a good solution on the producer side; you would need to replace the producer factory bean or the KafkaTemplate(s) with new overrides for the store locations.

I'll have opened an issue to make it easier there.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Thank you, I'll look into the suggestions you mentioned and thanks for opening a issue. – Kenneth Gunnerud Nov 24 '20 at 17:34
  • After checking a bit, how would this work in case of shutdown or other reasons to stop the concurrent container? They all publish an ConsumerStoppedEvent – Kenneth Gunnerud Nov 24 '20 at 18:18
  • We recently added a `reason` to the `ConsumerStoppedEvent`, but we don't currently have a reason for the auth failure ([I will add one now](https://github.com/spring-projects/spring-kafka/issues/1643)). In the meantime, you could try to create a consumer after the event to see if you get an auth exception. – Gary Russell Nov 24 '20 at 18:24