From the following issue at CrateDB GitHub page it seems it is not possible, i.e., the Kafka protocol is not supported by CrateDB. https://github.com/crate/crate/issues/7459
Is there another way to load data from Kafka into CrateDB?
Asked
Active
Viewed 392 times
1 Answers
1
Usually you'd use Kafka Connect for integrating Kafka to target (and source) systems, using the appropriate connector for the destination technology.
I can't find a Kafka Connect connector for CrateDB, but there is a JDBC sink connector for Kafka Connect, and a JDBC driver for CrateDB, so this may be worth a try.
You can read more about Kafka Connect here, and see it in action in this blog series:
- https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-1/
- https://www.confluent.io/blog/blogthe-simplest-useful-kafka-connect-data-pipeline-in-the-world-or-thereabouts-part-2/
- https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-3/
Disclaimer: I work for Confluent, and I wrote the above blog posts.
Robin Moffatt
- 30,382
- 3
- 65
- 92
-
1We also think a JDBC connection is the way to go. Is there any information on the performance penalty when using a JDBC connection instead of a connector using Kafka Connect? Just to know if at long term the development of one is worth it. – romulog Oct 12 '18 at 14:31
-
That's a question for someone with CrateDB experience, which I don't have, sorry. I guess it depends how efficient the JDBC implementation is vs a native API. – Robin Moffatt Oct 12 '18 at 14:44