0

I have installed Kafka with all default settings on my Ubuntu partition On my windows partition on the same machine, using python, I have followed the steps of this guide.

Producer1.py creates the topic 'First_Topic' in my Kafka But Consumer1.py does not show any information and times out after some time. When I access Kafka from my ubuntu terminal, am able to see the newly created Kafka but no messages are consumed/available.

So initially my doubt was that am not able to connect from Windows partition to Ubuntu, but as I can see that the Kafka topic has got created then it shouldn't be that case. Only change in code is the localhost which I've replaced as bootstrap_servers = ['DESKTOP-MyUbuntu:9092'] from the original bootstrap_servers = ['localhost:9092']

What changes do I need to do to read the kafka messages in my consumer1.py?

Jacob Lee
  • 4,405
  • 2
  • 16
  • 37
  • I have added the producer.flush and then as it was not working producer.close. Topics are getting created but without message –  May 10 '21 at 13:33
  • Unclear what you mean by "partition". If you're dual booting on only one machine, then Ubuntu wouldn't be running Kafka while Windows is running and the Ubuntu filesystem would not be available. If you're using a VM, then you need to change `advertised.listeners` on the broker – OneCricketeer May 10 '21 at 13:44
  • 1
    With WIndows10 WSL have downloaded Ubuntu and using the UBUNTU 20.0.4 app am able to install kafka on ubuntu –  May 10 '21 at 13:51
  • That's not a "partition", then, that's just a subshell/separate process. – OneCricketeer May 10 '21 at 13:54

1 Answers1

0

As mentioned in the comments, you'll need to modify advertised.listeners to expose your external desktop hostname or IP (run ipconfig and use the address given by the router) from the WSL2 environment. By default Kafka only accepts local connections (local, here, meaning within the same WSL hypervisor)

There's also some firewall or port forwards on WSL2 that needs adjusted - there's some answers here on it (use port 9092 for Kafka)

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Ok what you are saying does make sense, let me try it out. –  May 10 '21 at 14:00
  • Tried the options provided there but it's not working. Used netsh interface method, added a firewall rule still same result. Hyperv bridge method is not enabled in my machine so couldn't use that. –  May 10 '21 at 17:04
  • What would be the advertised.Listener set to? Can you provide an example –  May 10 '21 at 17:05
  • 1
    Had to set listener property,did not have to change the advertised. Listener value listeners=PLAINTEXT://[::1]:9092 worked –  May 10 '21 at 17:31
  • Setting both is generally the best idea, but since you did the netsh thing, the default advertised listener would work for the machine hostname / localhost – OneCricketeer May 11 '21 at 01:18
  • Ok I was gonna remove the netsh rule today and try it if that still works. –  May 11 '21 at 02:46