2

How many parallel write requests can I make to cassandra from a single connection in a nodejs process?

I am using: cassandra-driver@4.6.3 : https://www.npmjs.com/package/cassandra-driver I am not using batch... as i understand it is not recommended if all data is not in same partition.

My main concern is limitation if any from cassandra connection point of view..

From: Understand Cassandra pooling options (setCoreConnectionsPerHost and setMaxConnectionsPerHost)? I understand that 32k parallel write is approx limit

Erick Ramirez
  • 13,964
  • 1
  • 18
  • 23
Cpp crusaders
  • 117
  • 2
  • 10

2 Answers2

3

32k is the theoretical upper limit on the number of queries in-flight, but the real number heavily depends on the size of the cluster, how load is spread between nodes, etc. Sometimes, having more in-flight requests will put an additional load onto the cluster, and your latencies will be heavily increasing.

As I mentioned in that answer, it's recommended to test your setup & find optimal number of requests per connection. You can use NoSQLBench tool to find optimal configurations.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Thanks @Alex for quick revert, also in related matter, this issue has cm up bcse we are trying to avoid batching [since write is across partitions]. So hence one batch now splits into 500 individual writes. I understand if we want to use batching [across partitions] we should benchmark it right before using such an approach? – Cpp crusaders Jun 14 '22 at 12:05
  • basically one batch with 500 rows vs 10 batch requests with 50 rows each vs 500 individual writes.. – Cpp crusaders Jun 14 '22 at 12:08
  • 2
    if you can combine writes into the same partition into a batch, then it would be big improvement as it will be one write operation. if it's not possible, then do 500 individual writes - just make sure that you're using prepared statements with token aware load balancing policy (default one) – Alex Ott Jun 14 '22 at 12:29
0

For the Node.js driver, the maximum requests per connection for native protocol v3 or newer is 2048 by default (for older native protocols v1 and v2 it's just 128).

It's not 32K as you quoted. The other post you linked isn't relevant to your question since it applies to the Cassandra Java driver.

For details, see Connection pooling with the Node.js driver. Cheers!

Erick Ramirez
  • 13,964
  • 1
  • 18
  • 23