Creating an asychroneous api for nodeJS (or browser)

Question

Most NodeJS programmers know why it's bad to block NodeJS's single-threaded event loop. For example, when reading a file, we know it's better to use fs.readFile instead of fs.readFileSync. My question is: how does one go about creating an asyn API in the first place if the underlying task to be performed by the API is inherently synchronous? Another words, how do I go about creating an API such as fs.readFileSync, and not use the event-loop thread to perform the underlying task? Do I have to go outside of NodeJS & Javascript to do this?

Answer 1

how does one go about creating an asyn API in the first place if the underlying task to be performed by the API is inherently synchronous?

In node.js, there are the following choices for creating your own asynchronous operation:

1. Base it on existing async operations (such as fs.readFile()). If your main operation itself is synchronous and can only be done synchronously in node.js, then this option would obviously not solve your problem.
2. Break your code into small chunks using setTimeout(), setImmediate() or nextTick() so that other things can interleave with your operation and you won't block other things from sharing the CPU. This doesn't prevent CPU usage, but it does allow sharing of the CPU with other operations. Here's an example of iterating over an array in chunks that doesn't block other operations from sharing the CPU: Best way to iterate over an array without blocking the UI.
3. Move your synchronous operation into another process (either another node.js process or any other process) that can do its thing and then communicate back asynchronously when done using any appropriate inter-process communication mechanism (TCP, stdio, etc...).
4. Write a node.js plug-in where you have the ability to use native OS threading or native IO events and you can then create a new operation in node.js that has an async interface.

Do I have to go outside of NodeJS & Javascript to do this?

Items #1, #2, #3 above can all be done from Javascript. Item #4 involves create a native code plug-in (where you can have access to native threading).

Another words, how do I go about creating an API such as fs.readFileSync

To truly create an API like fs.readFileSync() from scratch, you would have to use either option #3 (do it synchronously, but in another process and then communicate back the result asynchronously) or option #4 (access OS services at a lower level than is built into node.js via a native code plug-in).