1

My N1QL query returns around 3.5 million documents. Is there a way in which I can stream those documents to the client? I see there is a kafka connector available in couchbase but seems like it works on DCP wherein all the mutations to the documents are send to kafka. I have a fixed N1QL query and want the query results to be streamed.

Matthew Groves
  • 25,181
  • 9
  • 71
  • 121
Maninder Chhabra
  • 103
  • 2
  • 11
  • 2
    Have you looked into using Reactive/Async - https://docs.couchbase.com/java-sdk/current/howtos/n1ql-queries-with-sdk.html#reactive-and-async-apis – Matthew Groves May 11 '22 at 14:44
  • N1QL streams the results when it ready. https://couchbase.live/examples/basic-java-query-named-param checkout. Also n1ql can project document key, stream and use SDK to Fetch document from kv parallel Reactive/Async – vsr May 11 '22 at 15:24
  • We used eventing with cURL posts to stream couchbase document mutations to our services. It works pretty well. Maybe that is an alternative for you approach, instead of quering 3.5 million documents at once, why now stream each document mutation with eventing – Siraf Aug 05 '22 at 14:15

0 Answers0