How can I make an item in an array of tasks await completion of others?

Question

I have a long array of objects containing:

1) Name of the process (the 'name') 2) A function (the 'task') 3) The arguments for that function (the 'arguments')

Some of the tasks are dependent on other tasks being completed before they can be carried out. I want them to dynamically await the completion of any dependencies.

What I was hoping to do was to have an async function providing the name argument for the functions that are dependent on other parts. In that way a function would be on hold until the task can be executed. I need it to be dynamic, no 2 or 3 lists executed separately. There may be tasks depending on other dependent tasks.

A list of tasks:

let tasks = [
    {name: 'numerator', task: insert, arguments: ['part', 'UK', '2010']},
    {name: 'divide', task: calculate, arguments: ['UK', '2010', await waitFor('numerator'), await waitFor('denominator')]},
    {name: 'denominator', task: insert, arguments: ['whole', 'UK', '2010']}
];

A loop:

tasks.forEach(d => {
    d.task(d.arguments);
}

Functions:

async function waitFor(taskName) {
    await tasks.find(d => d.name === taskName).task;
    return taskName;
}

async function insert(mode, country, year) {
    //Connect to database, sum up raw data and insert
}

async function divide(country, year, upper, lower) {
    //Connect to database, retrieve the values, divide one by the other and insert
}

Now, I know very well that the above solution does not work. I see two problems: 1) The waitFor function reference the tasks array before it is initialized 2) Promises do not work the way I hoped they would. Execution of the dependent task will not magically be delayed until other tasks are done.

Some may say that this is a stupid way of doing things. I have inherited code where there is a set order and await for every single task and that is very time consuming. I wish to create a system where tasks can be added and everything is automatically coordinated. So the question is:

How should the waitFor function be written and how should it be used in the tasks array for this to work?

Very grateful for help with this.

Does it necessarily need to be a `taskList` (that would require topological sorting, rejection of invalid dependencies, etc), or could you just write normal code? — Bergi, Jul 26 '19 at 15:57
@bergi Thanks for your time. I think any code that does the trick of executing a bunch of functions asynchronously with automatic coordination is fine. I'm sorry about writing 'taskList' which is the actual variable name of the array in my code. The idea is just to replace a soviet style form-a-queue-and-await system that is slow and hard to maintain. — K. Nordqvist, Jul 26 '19 at 16:36
So do you need the automatic coordination, being able to wait for things that are declared below in the list? See my answer for the approach where you just want to optimise the queue-await code, allowing concurrency where possible. Btw, if you could post your actual code (with more than just three calls), it might be better to demonstrate. — Bergi, Jul 26 '19 at 16:40

score 1 · Answer 1 · answered Jul 26 '19 at 16:03

1

I think you are looking for

const numeratorPromise = insert('part', 'UK', '2010');
const denominatorPromise = insert('whole', 'UK', '2010');
const dividePromise = Promise.all(['UK', '2010', numeratorPromise, demoniatorPromise])
                      .then(args => calculate(...args));

// await dividePromise - or whatever you want to do with the result

You can build arbitrarily many promises in this style, with arbitrary dependencies on previously created promises, and they will run as concurrent as possible. Promise.all ensures the dependencies are fulfilled. Then just wait for the single, final result promise.

answered Jul 26 '19 at 16:03

Bergi

630,263
148
957
1,375

That requires that a task always occurs after its dependencies ...thats not the case in the OPs code. – Jonas Wilms Jul 26 '19 at 16:08
1

@JonasWilms Yes, that's why I wrote "*dependencies on previously created promises*". But from "*I have inherited code where there is a set order and await for every single task and that is very time consuming*" I judged that the OP doesn't actually have an array of arbitrarily ordered tasks, he just wants a solution that lets him run multiple things concurrently and is able to adopt the order. – Bergi Jul 26 '19 at 16:13
This would work but I prefer not to have to deal with the order of things if possible. The order is there but annoying to maintain. – K. Nordqvist Jul 29 '19 at 14:51
@K.Nordqvist Well you have to deal with order at least a bit, as to avoid introducing circular dependencies. If your function is getting hard to maintain, you might want to consider refactoring it in multiple smaller parts that can run independently. – Bergi Jul 29 '19 at 14:58
@K.Nordqvist If you really want to go with a datastructure into which you can insert computations arbitrarily, I'd recommend writing a [topological sort](https://en.wikipedia.org/wiki/Topological_sorting) algorithm that establishes an order (and, importantly, throws an error if you messed up - Jonas' solution would just hang in that case), then build a promise for each task just like in my answer. – Bergi Jul 29 '19 at 15:00

Jonas Wilms · Accepted Answer · 2019-07-26T16:23:15.883

1

I'd just return a promise from waitFor, then resolve that when the related task is done. Evaluating a task awaits all its arguments first:

 const byName = {};

  let tasks = [
   {name: 'numerator', task: insert, arguments: ['part', 'UK', '2010']},
   {name: 'divide', task: calculate, arguments: ['UK', '2010', waitFor('numerator'), waitFor('denominator')]},
  {name: 'denominator', task: insert, arguments: ['whole', 'UK', '2010']}
 ];


 function waitFor(name) {
   if(byName[name]) {
      return byName[name].done;
   } else {
      const task = byName[name] = { name };
      return task.done = new Promise(resolve => task.resolve =  resolve);         
   }
 }

 function runTask({ name, task, arguments }) {
   waitFor(name); // cause promise to be set up, dont await!

   byName[name].resolve(
    Promise.all(arguments)
      .then(args => task(...args))
   );
 }

 tasks.forEach(runTask);

Make sure that there are never cyclic dependencies, that will cause a deadlock.

edited Jul 26 '19 at 16:23

answered Jul 26 '19 at 16:07

Jonas Wilms

132,000
20
149
151

1

You'll want to initialise `byName` before you create the `tasks` array which calls the hoisted `waitFor` functon :-) – Bergi Jul 26 '19 at 16:14
1

Don't do `.then(byName[name].resolve)`, that's the [`Promise` constructor antipattern](https://stackoverflow.com/q/23803743/1048572?What-is-the-promise-construction-antipattern-and-how-to-avoid-it) which doesn't handle rejections properly. At least do `task.resolve(Promise.all(arguments).then(args => task(...args)));` if you must construct deferreds for not-yet-started tasks. – Bergi Jul 26 '19 at 16:17
Btw, `arguments` is not a valid parameter name in strict mode – Bergi Jul 26 '19 at 16:18
@bergi agreed & edited, the OP shouldnt use `arguments` as a property name at all though ... – Jonas Wilms Jul 26 '19 at 16:25
1

@JonasWilms That is a very elegant solution, thanks a lot! FYI almost a bit too elegant, the compact way of writing for example `const task = byName[name] = { name }`confused me at first and almost prompted me to post another question :) – K. Nordqvist Jul 29 '19 at 14:41

How can I make an item in an array of tasks await completion of others?

2 Answers2