6

I am trying to execute following array (avoid callbackHell) of functions(sync/async), in a sequential order, implementing function runCallbacksInSequence (I need to implement my own function to understand how callbacks work and avoid using Async.js).

Here is what I have so far. The function runCallbacksInSequence works well till it gets the same callback more than once. It stops and does not continue to execute the next callback. Ideally if it gets the same callback more than once it should not execute it second time and continue with the next callback.

If you have any ideas let me know what I am doing wrong and how I can fix it. - no promises and async/await

function first(cb) {
  setTimeout(function() {
    console.log('first()');
    cb(null, 'one');
  }, 0);
}

function second(cb) {
  setTimeout(function() {
    console.log('second()');
    cb(null, 'two');
  }, 100);
}

function third(cb) {
  setTimeout(function() {
    console.log('third()');
    cb(null, 'three');
  }, 0);
}

function last(cb) {
  console.log('last()');
  cb(null, 'lastCall');
}

const cache = {};

function runCallbacksInSequence(fns, cb) {
  fns.reduce(
    function(r, f) {
      return function(k) {
        return r(function() {
          if (cache[f]) {
            return;
            // f(function(e, x) {
            //   e ? cb(e) : k(x);
            // });
          } else {
            cache[f] = f;
            return f(function(e, x) {
              return e ? cb(e) : k(x);
            });
          }
        });
      };
    },
    function(k) {
      return k();
    }
  )(function(r) {
    return cb(null, r);
  });
}

const fns = [first, second, third, second, last];

runCallbacksInSequence(fns, function(err, results) {
  if (err) return console.log('error: ' + err.message);
  console.log(results);
});
John John
  • 1,305
  • 2
  • 16
  • 37
  • Possible duplicate of [Executing callbacks in sequential order](https://stackoverflow.com/questions/56471930/executing-callbacks-in-sequential-order) – Herohtar Jun 11 '19 at 20:56

3 Answers3

2

Your function chaining depends on the call to k(). Therefore in your cache logic:

if (cache[f]) {
    return;
} else {
    // ...

The chain breaks.

What you want instead is this:

if (cache[f]) {
    return k();
} else {
    // ...

Alternative Implementation

One of the problems with the nested function implementation is that it is hard to reason about due to multiple nesting scopes (and multiple functions being juggled at once (r, f, k, cb).

A simpler approach to this is rather than trying to programmatically build callback hell you can use a queue instead (which is what async.js does). The idea is simple, pop() or shift() functions from an array until the array is empty:

function runCallbacksInSequence(fns, cb) {
    let result = [];
    let cache = {};

    function loop () {
        if (fns.length > 0) {
            let f = fns.shift(); // remove one function from array

            if (cache[f]) {
                loop(); // skip this round
                return;
            }

            cache[f] = f;
            f(function(err, val) {
                if (!err) {
                    result.push(val); // collect result
                    loop();
                }
                else {
                    // Handle errors however you want.
                    // Here I'm just terminating the sequence:
                    cb(err, result);
                }
            });
        }
        else {
            cb(null, result); // we've collected all the results!!
        }
    }

    loop(); // start the loop
}

As you can see, it's fairly easy to implement any flow logic with this structure. We can easily implement things like waterfall, parallelLimit etc. by controlling how we keep track of results and how many functions we remove from the array per iteration.

slebetman
  • 109,858
  • 19
  • 140
  • 171
  • I am wondering why when I call `const fns = [first, second, second];` the outer callback returns `undefined` ? – John John Jun 10 '19 at 05:21
  • Hmm. That may be due to calling `k()` without a value. It is the same as calling `k(undefined)`. Your construction is still callback hell but done programatically so it's kind of hard to reason about – slebetman Jun 10 '19 at 05:22
  • I guess I would have to think about it in a different way to solve it – John John Jun 10 '19 at 05:26
  • @JohnJohn FWIW. I originally misread your question and offered you an alternate way to do this using a queue rather than trying to construct nesting functions. If you are interested I can either undelete that answer or copy it into this one – slebetman Jun 10 '19 at 05:29
  • Yes for sure. I would look at it for inspiration! – John John Jun 10 '19 at 05:32
  • it looks much better! I have few issues to work out. I have a question for you. How do we look up the implementation of async.js . I tried to look it up but it looked very complicated – John John Jun 10 '19 at 07:42
  • 1
    I wrote this answer a long time ago before async.js: https://stackoverflow.com/questions/4631774/coordinating-parallel-execution-in-node-js/4631909#4631909. After async.js became available I suspected that they were inspired by my code because the feature sets they implemented could easily be modified from it. I didn't actually read the implementation until just now and it looks like they just keep track of an index into the functions array (see lib/internal/iterator.js line 4) instead of using `shift()`... – slebetman Jun 10 '19 at 08:14
  • 1
    ... as for how to read production code, you just need practice. After some years and a handful of projects you will learn to read code by jumping around function/method definitions. An IDE with "go to definition" functionality helps but not necessary (I found the iterator implementation of async.js using github on my browser) – slebetman Jun 10 '19 at 08:16
  • I changed a bit the code to extract only the result of the last function called as per task I am trying to accomplish. The only problem I have is handling the errors. – John John Jun 10 '19 at 08:41
  • I looked into fork implementation and it solves in a very simple way a very complex issue. it is really thinking out of the box. I have a question though. Is there a way to control how many times a callback is executed within a function? – John John Jun 13 '19 at 04:22
  • as an example ```function last(cb) { console.log('last()'); cb(null, 'lastCall'); cb(null, 'lastCall'); }``` – John John Jun 13 '19 at 04:23
  • we can controll how many times to call the function from the array but what is going on inside the function I really do not understand how to control it. – John John Jun 13 '19 at 04:27
1

I guess with implementation based on cache you may omit doubled step with a direct k() invocation.

return;
if (cache[f]) {
  return;
  // f(function(e, x) {
  //   e ? cb(e) : k(x);
  // });

Idea:

if (cache[f]) {
  return k(function(e, x) {
    return e ? cb(e) : k(x);
  });
MartenCatcher
  • 2,713
  • 8
  • 26
  • 39
1

Your code is a little bit hard to read for me. So here is the alternative solution:

<script>
  // The data

  function first(cb) {
    setTimeout(function () {
      console.log('first()');
      cb(null, 'one');
    }, 0);
  }

  function second(cb) {
    setTimeout(function () {
      console.log('second()');
      cb(null, 'two');
    }, 100);
  }

  function third(cb) {
    setTimeout(function () {
      console.log('third()');
      cb(null, 'three');
    }, 0);
  }

  function last(cb) {
    console.log('last()');
    cb(null, 'lastCall');
  }

  const fns = [first, second, third, second, last];

  // We need hash function to create the identifyer of the function
  function hashCode(str) {
    return Array
      .from(str)
      .reduce((s, c) => Math.imul(31, s) + c.charCodeAt(0) | 0, 0);
  }
  const cache = [];

  function reducer(accumulator, currentFunction) {
    // Take the functon string representation to detect "the same function"
    const hash = hashCode(currentFunction.toString());
    // Process the result of the current function and call the next one.
    // We use "reduceRight" so `accumulator` is the next function in the chain.
    const cb = function (fp, result) {
      console.log(result);
      // Cache the result;
      cache[hash] = result;
      accumulator();
    }
    // Run just a callback if we already have the result of the current function
    return () => cache[hash] ? cb(null, cache[hash]) : currentFunction(cb);
  }

  fns.reduceRight(reducer, () => { })();
</script>

Result:

first()
one
second()
two
third()
three
two
last()
lastCall

If you do not want to process the cached result at all, then replace the call to the callback with the call to the accumulator directly.

return () => cache[hash] ? cb(null, cache[hash]) : currentFunction(cb);

replace with:

return () => cache[hash] ? accumulator() : currentFunction(cb);

Result:

first()
one
second()
two
third()
three
last()
lastCall

Solution without cache

It is much cleaner:

<script>
  // Use the same data as in the example with cache

  function reducer(accumulator, currentFunction) {
    const cb = function (fp, result) {
      console.log(result);
      accumulator();
    }
    return () => currentFunction(cb)
  }

  fns.reduceRight(reducer, () => { })();
</script>

Result:

first()
one
second()
two
third()
three
second()
two
last()
lastCall
Andriy Kuba
  • 8,093
  • 2
  • 29
  • 46
  • 1
    the code looks very interesting but it repeats the results. I will upvote it for the effort! – John John Jun 10 '19 at 08:56
  • 1
    Yes, it repeats the result to show that step was done. I.e. You can process the cached result (aka log the result of the function) without running function itself. Anyway, it is easy to change the code so it does not repeat the result at all. Please, look at the updates. – Andriy Kuba Jun 10 '19 at 09:51