12

Play Framework advises to relay blocking IO to an appropriate sized thread pool, as in:

https://www.playframework.com/documentation/2.5.x/ThreadPools

This is the case for relational database access because there are no non-blocking JDBC drivers available (with few exceptions).

I am currently learning about Node.JS and I couldn't figure out how this is handled in node. I didn't see any need to code thinking about thread pools in node.

So, are the relational database drivers used in node.js able to do non-blocking IO? Or are these computations being relayed to some kind of worker threads behind the scenes?

On a broader sense: what is the correct way to code a node.js app that is very DB (relational) intensive?

Renan
  • 1,705
  • 2
  • 15
  • 32
  • There is an article about that which you [might enjoy](https://engineering.linkedin.com/play/play-framework-async-io-without-thread-pool-and-callback-hell) – AME Apr 25 '16 at 07:34
  • Take a look at this question, it may help clear up your understanding of how node works under the hood, but to answer your question, all node db drivers that I know of use non-blocking io interfaces/the libuv async event loop... you shouldn't have to worry about implementing that at the application level: http://stackoverflow.com/questions/14795145/how-the-single-threaded-non-blocking-io-model-works-in-node-js – photoionized May 02 '16 at 18:28

2 Answers2

3

Node is single threaded so there are no user thread pools[1]. Instead you need to scale horizontally with more Node servers. And you can do that within a Node app: https://devcenter.heroku.com/articles/node-concurrency

And on another note, I've had good success with the async-JDBC-ish postgresql-async driver. I've used it with jdub-async and scalikejdbc. Here is a blog I wrote on using it with scalikejdbc: https://www.jamesward.com/2015/04/07/reactive-postgres-with-play-framework-scalikejdbc

[1] User code runs single threaded (but you can use web workers to have threads) however libuv is multi-threaded. Read more: How the single threaded non blocking IO model works in Node.js

James Ward
  • 29,283
  • 9
  • 49
  • 85
  • mysql module for node.js can use connection pool as per https://www.npmjs.com/package/mysql#pooling-connections Maybe node.js uses internal thread pools for this? – Renan May 07 '16 at 14:17
  • Here's another complementary reference stating that node.js is multi-threaded (your code, however, runs in a single thread): http://rickgaribay.net/archive/2012/01/28/node-is-not-single-threaded.aspx – Renan May 07 '16 at 14:27
  • Yeah, libuv has a thread pool so network IO can have concurrency. But even though you have a connection pool, that doesn't mean you get a thread per connection. Check out: https://www.future-processing.pl/blog/on-problems-with-threads-in-node-js/ – James Ward May 08 '16 at 17:33
2

I think you basically answered your own question: in nodejs you don't have to code in terms of thread pools or so. DB thread pools in Play are inherent to Java JDBC API. Pure nodejs DB drivers are asynchronous by design. The architecture of a nodejs wrapper driver depends on that of a wrapped library.

The answer to the broader question is:

There is not much difference between how you code DB intensive apps in nodejs or java, as most probably your bottleneck will be persistent storage behind your DB regardless of platform. But in asynchronous architectures:

  1. it is more natural to design a system that doesn't overwhelm your DB with too much load

  2. in case of a DB slowdown, the application itself usually will not demand more system resources

A good DB driver will let you achieve the points above with managed connection pools, per-query time-outs, per-connection query-queues. Though some of those could also be a feature of the native DB interface.

Tair
  • 3,779
  • 2
  • 20
  • 33