Node.js asynchronous control flow with 3 processes working on shared data

Question

I have node.js app with 3 functions, each one has own interval (or can by fired by another event):

processOrders
clearExpiredOrders
clearCancelledOrders

Functions are working on shared data, so only one of them can access this data at the time.

When there is a collision, it should wait until the actual function ends and then run second function immediately, but only once.

For example, if there is processOrders running and clearCancelledOrders is triggered 100 times, it will run clearCancelledOrders only once after processOrders finishes its work.

How would you solve this in node.js? Or do you have another idea how to solve this task?

You need a message queue and your requirements are still ill-defined. Please research more before posting here. Try Kue. Also look at the design of RabbitMQ or ZeroMQ. — djechlin, May 01 '14 at 23:18
Specifically it's incomplete because you don't define when clearCancelledOrders becomes reenabled. Proper concurrency control here makes this a huge question. For "getting started," message queueing should give you something to google, and if you have a start may be okay to post a question here. — djechlin, May 01 '14 at 23:21
I will look at it, originally I expected something like promises or event based solution where I can't exactly imagine how it should work for my task.. but I will do research about message queues — Jan Jůna, May 01 '14 at 23:51

score 1 · Answer 1 · edited May 23 '17 at 12:28

Here is an example of how to do this very simply with promises:

var p  = Promise.resolve(); // assuming Bluebird promises or Node 0.11.13+ with Promises.
                            // create a new empty resolved promise.

function doFirst(act){
   return p = p.then(function(){
      return someFirstAction(act); // assumes someFirstAction returns a promise
   })
}
function doSecond(act){
   return p p.then(function(){
      return someSecondAction(act); // assumes someFirstAction returns a promise
   })
}

What this does is queue the operations on the single chain. When it resolves the chain resolves. It also returns the returned promise, so you can unwrap it and get the value.

For example:

doFirst(1);
doSecond(2);

// some point in the future
doFirst(3).then(function(value){
   // both doFirst(1) and doSecond(2) are done here
   // the value argument is the resolution value of doFirst(3)
});

If you're unsure on how to convert your API to promises - see this question.

Since, you also want to limit the number of times a particular action is run, you can create special methods for it:

doFirst.queued = false;
function doFirst(act){
   if(doFirst.queued) return; // already queued
   doFirst.queued = true;
   return p = p.then(function(){
      return someFirstAction(act).finally(function(){ 
          doFirst.queued = false;
      }); 
   })
}
doSecond.queued = false;
function doSecond(act){
   if(doSecond.queued) return; // already queued
   doSecond.queued = true; 
   return p = p.then(function(){
        return someSecondAction(act); // assumes someFirstAction returns a promise
   }).finally(function(){
        doSecond.queued = false;
   });
}

Queuing by `.then()` is not enough here. OP wants that "*if clearCancelledOrders is triggered 100 times [during some other processing], it will run clearCancelledOrders only once [after the other task]*" — Bergi, May 04 '14 at 13:07
Oh, I totally missed that although it's huge and bold, my bad. Lemme fix that. — Benjamin Gruenbaum, May 04 '14 at 13:14

Node.js asynchronous control flow with 3 processes working on shared data

1 Answers1