25

I have just started learning Kafka. So trying to build a social media web application. I am fairly clear on how to use Kafka for my backend ( communicating from backend to databases and other services).

However, I am not sure how should frontend communicate with backend. I was considering an architecture as: Frontend -> Kafka -> Backend.

Frontend acts as producer and backend as consumer. In this case, frontend would supposedly have all required resources to publish to Kafka broker (even if I implement security on Kafka). Now, is this scenario possible:

Lets say I impersonate the frontend and send absurd/invalid messages to my Kafka broker. Now I can handle and filter these messages when they reach to my backend. But I know that Kafka stores these messages temporarily. Wouldn't my Kafka server face DDOS problems if such "fake" messages are published to it in high volume, since it is gonna store them anyway as they dont get filtered out until they actually get consumed by backend?

If so, how can I prevent this?

Or is this not a good option? I can also try using REST for frontend/backend communication and then Kafka will be used from backend to communicate with database(s) and other stuff.

Or I can have a middleware (again, REST) that detects and filters out such messages.

user3271166
  • 573
  • 1
  • 6
  • 17
  • 1
    Care to explain the reason for downvote? This is a genuine question I have that I cant find answer to even with lots of researching. What else can I do to improve this question? – user3271166 Dec 11 '17 at 05:16
  • It was probably downvoted, and voted for closure, because you are not asking a specific question about a specific programming problem as required and outlined in the Help Center. It is very broad and asks for opinion with no clear outline of what you are doing now, which is not allowed on this fact based platform. If this were not a weekend late evening, I'm sure it would have been closed by now. – Rob Dec 11 '17 at 09:53
  • 1
    All I asked was: `If my frontend posts messages to Kafka as a producer, how do I prevent unwanted DDoS attacks or such`. I am not sure how was it broad, but I'll try word it better in future. – user3271166 Dec 12 '17 at 03:55

2 Answers2

9

Easiest way is to have the front end produce to the Kafka REST Proxy

See details here https://docs.confluent.io/1.0/kafka-rest/docs/intro.html

That way there is no kafka client code required in your front end and you can use HTTP(S) with standard off the shelf load balancers, and API Management tools.

Hans Jespersen
  • 8,024
  • 1
  • 24
  • 31
  • Thanks for the answer. That indeed does seem to resolve the issue. But I dont see how might it be different than me implementing my own "middleware". So Frontend -> My Middleware -> Kafka -> Backend. Plus I can use whatever technologies and methods I prefer instead of being limited to REST (in case I decide to switch to something else other than REST in future). Am I on right track in my above thoughts? – user3271166 Dec 11 '17 at 05:19
  • 1
    If you want to use something other than REST then look at Kafka Connect. There are connectors for MQTT, AMQP, websockets, and many more https://www.confluent.io/product/connectors/. If you really want to build your own middleware you can but the beauty of Kafka Connect is that a lot is already written and it’s all open source so you can start from something you already know works. – Hans Jespersen Dec 11 '17 at 05:47
0

Could you not consider the other direction, to use Kafka as a transport system for updating assets available to frontend ? This has been proposed for hybrid React / NodeJS/Express solutions.

Gunnar Forsgren - Mobimation
  • 1,381
  • 10
  • 14