0

On my standalone windows local machine i have apache kafka (2.11-2.0.0) along with my multi microservice app (spring java and python). For more than a year kafka ran fine. I never changed the settings nor touched it.

In the last week or so I noticed I was getting kafka issues in my app's logs

In python:

No broker metadata found in MetadataResponse -- ignoring.

In spring java:

Error while fetching metadata with correlation id 3: {abc.process.file.request=LEADER_NOT_AVAVAILABLE}

both errors were just spamming in the logs. Initially I thought this was app issue and I tried everything I could including wiping out the code and rebuilding from scratch only to get the same error

Only after a long time of researching I found a workaround to modify kafka's server properties with:

port = 9092
advertised.host.name = localhost

App now working as normal but my question is why would I have to change server settings when it ran fine for a year plus.

The only thing I can think of is that for the last week I've (unsuccessfully) tried using WSL2 to install docker which included playing around with network config but i don't see how that would impact a localhost run, I'm not connecting to anything outside my PC

Any help?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
CaptainObv
  • 360
  • 1
  • 4
  • 15

1 Answers1

0

You mentioned you are using Docker... Are you not using a consistent Docker image tag? Therefore defaults to :latest? And therefore will pull the most recent Kafka version, where port and advertised.host.name have both been deprecated properties (and may now be removed)? (You should have been using listeners setting all along).

Or, have you not mounted persistent storage to that container, or otherwise mounted a folder like /tmp that may have been cleared.

WSL2 has had some upgrades over the last year, or so, therefore may have reset some state itself, including Docker images.

The above aren't really Kafka problems, but rather Docker best practices for having consistent, deterministic environments.

playing around with network config

See Connect to Kafka running in Docker ; You should use Docker for Windows, and enable Docker Integration in the GUI, rather than apt install docker, manually , for example.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245