3

I'm using Cassandra with nodejs to fetch big table with eachRow.

I need to insert data on each row, but for some reason it's not waiting for the query and it finish before it's done.

client.eachRow(query, [], { prepare: true, autoPage : true, fetchSize: 500 }, function(index, row) {
     // DB query / insert or update
, function(err, result) {
    // Finish all rows.
});

Any suggestions?

Rafael Mor
  • 626
  • 1
  • 7
  • 21

1 Answers1

5

simply go through official cassandra-driver doc https://www.npmjs.com/package/cassandra-driver and figure out this line

The #stream() method works in the same way but instead of callback it returns a Readable Streams2 object in objectMode that emits instances of Row.

client.stream(query, [ 'abc' ])
  .on('readable', function () {
    // 'readable' is emitted as soon a row is received and parsed
    var row;
    while (row = this.read()) {
      console.log('time %s and value %s', row.time, row.val);
    }
  })
  .on('end', function () {
    // Stream ended, there aren't any more rows
  })
  .on('error', function (err) {
    // Something went wrong: err is a response error from Cassandra
  });
shivshankar
  • 2,067
  • 1
  • 18
  • 28
  • Thanks, but the "end" will not finish until all db queries done from the "readable"? – Rafael Mor Nov 23 '17 at 19:08
  • no you can pause or exhaust stream anytime. please read more about Readable Streams https://nodejs.org/en/blog/feature/streams2/ – shivshankar Nov 23 '17 at 19:11
  • Yes I've found this: "The #stream() method works in the same way but instead of callback it returns a Readable Streams2 object in objectMode that emits instances of Row. It can be piped downstream and provides automatic pause/resume logic (it buffers when not read)." Can you please give me a example code how to use the pause / continue? – Rafael Mor Nov 24 '17 at 14:47
  • Will you post the GIST? – Rafael Mor Nov 26 '17 at 04:23
  • I never work with Cassandra, answer is bias on stream only. try https://gist.github.com/shivshankar3578/ce08a39b7d1f9155f8f8b7df6c70f8d7 – shivshankar Nov 26 '17 at 05:41
  • I've test it, the stop working but the code inside the timeout not running, it's like the stop not really stop. – Rafael Mor Nov 26 '17 at 10:36
  • make sure query return more then one row. – shivshankar Nov 26 '17 at 17:29