1

I am using Pivotal Cloud Foundry for deploying my microservices. I am building an eventservice which uses Kafka for messaging. Is Kafka available in PCF marketplace or is it possible to run kafka on PCF or should it be hosted externally?

Punter Vicky
  • 15,954
  • 56
  • 188
  • 315
  • I guess I would point you to this answer. https://stackoverflow.com/questions/34745938/how-to-use-kafka-and-storm-on-cloudfoundry Something may have changed in the last year. – DoomVroom Aug 14 '17 at 18:20
  • Thanks @DroomVroom. This seems to be early last year. I wanted to see if anything changed after that as you mentioned since it seems to have been in the works. – Punter Vicky Aug 14 '17 at 18:21

1 Answers1

2

The options from Amit's response here are still valid. The major difference is that the CF persistence proposal that he linked to has been completed and the feature is called Volume Services. By using Volume Services, you can have a persistent store mounted into your application container. This would, in turn, allow you to deploy Kafka as an application on PCF and not lose any data.

That said, you would still need a service broker if you want to integrate Kafka into the marketplace. That is optional though, as you could create user provided services instead, and that doesn't require a broker.

Hope that helps!

Daniel Mikusa
  • 13,716
  • 1
  • 22
  • 28
  • Thank you very much , Daniel. Can Volume Services be extended to installing a relational DB or NoSQL DB in PCF? – Punter Vicky Aug 15 '17 at 11:40
  • I don't believe that is the intent. My understanding is that Volume Services targets the use case of legacy applications that must write to a file system. An example of this would be Wordpress. – Daniel Mikusa Aug 15 '17 at 11:49
  • Thanks Daniel , one last questions. Is there any documentation on how to install Kafka on PCF with the help of volume services? I tried searching but wasn't able to find one. – Punter Vicky Aug 15 '17 at 15:44
  • I don't know of anything. Volume Services are pretty new, so you might find a few demos on how to use that. You'd have to adapt that to deploying Kafka on PCF. If you're going the Docker route, that might be easier cause what applies there should generally apply for PCF. – Daniel Mikusa Aug 15 '17 at 17:35
  • In case you need a service broker for integration in the PCF marketplace, this is an implementation for provisioning topics on a shared cluster (no dedicated clusters): https://github.com/1123/kafka-service-broker – user152468 Jan 10 '20 at 08:16