Javascript implement async to get result of long-running process into array

Question

I have an array of files that I am adding data to which conceptually works like this:

let filearray = ['file1.txt', 'file2.txt', 'file3.txt'];
newarray = [];
for (let f of filearray) {
  newstuff = 'newstuff';
  newarray.push([f, newstuff]);
}
console.log(newarray)
// Returns expected array of arrays

However, what I need to do is make newstuff = slow_promise_function(f); that involves lots of processing. How do I get the value from that promise function into the array?

Ideally I'd like to use the new async feature in ES2017.

Update:

These answers are helping me understand the problems (and solutions):

Folks could help you a lot better if you showed examples of what your long running asynchronous code is that involves lots of processing. And, BTW, the answer is completely different if your long running code is not actually asynchronous. Then, to make it actually asynchronous, you will likely have to put it in another process. — jfriend00, Aug 12 '17 at 02:39
Multiple functions. Some parse / analyze file streams, some do heavy computing. I purposefully left it general as I struggle to wrap my brain around JavaScript's async nature. In python I say `do a, then do b, then do c` and it all just works (even if it's slow). — Trees4theForest, Aug 12 '17 at 12:26
Well, non-specific questions like this make us "guess" what you're actually trying to do and we can't really help you very well. The answer is entirely different depending upon what exactly you're trying to do. Per stack overflow guidelines, I'd say this is a pretty poor question - far too broad and can't really be answered very well without guessing. Please show us actual code situations you want to see a solution for. — jfriend00, Aug 12 '17 at 17:02
For example, stuff that is slow because of heavy computing, but otherwise synchronous is not async at all and will not benefit at all from using promises or async. In fact, if you want your server to be able to do other things while heavy computing is going on, then you have to move that computing to a different worker process. If it's just asynchronous file or network I/O, then that can be done with promises, but how to do it with promises depends upon the nature of the operations. — jfriend00, Aug 12 '17 at 17:05

score 4 · Answer 1 · answered Aug 11 '17 at 22:57

4

You could use Promise.all which returns a single Promise that resolves when all of the promises have resolved:

let loadData = async () => {
  let filearray = ['file1.txt', 'file2.txt', 'file3.txt'];

  try {
    let ops = filearray.map(f => slow_promise_function(f));
    let newarray = await Promise.all(ops);

    // TODO: use newarray
  } catch (err) {
     console.log(err);
  }
}

loadData();

answered Aug 11 '17 at 22:57

alexmac

19,087
7
58
69

doesnt array.map override the element values instead of appending? – Trees4theForest Aug 12 '17 at 01:08
2

`array.map` creates a new array with the results of calling `slow_promise_function` on every file in `filearray`. `ops` contains the list of unresolved promises, when `newarray` after calling `Promise.all`, the list of values. – alexmac Aug 12 '17 at 09:08

Noah Freitas · Answer 2 · 2017-08-11T23:15:40.573

2

async/await is a nice way to accomplish this, as you suspected:

console.log('Starting...');

let files = ['file1.txt', 'file2.txt', 'file3.txt'];

Promise.all(files.map(async f => [f, await slow_promise_function(f)]))
       .then(files => console.log('got results: ', files));

function slow_promise_function(file) {
    return new Promise(res => setTimeout(_ => res(`processed ${file}`), 500));
}

edited Aug 11 '17 at 23:15

answered Aug 11 '17 at 22:56

Noah Freitas

17,240
10
50
67

2

In your example, the files will be loaded sequentially. – alexmac Aug 11 '17 at 23:04
@alexmac that is very true. I probably mistakenly assumed that was a requirement. Thanks. If it's not, then `Promise.all` is definitely the way to go, as your answer uses. – Noah Freitas Aug 11 '17 at 23:06
I think I preferred the original (sequential) as that's how my brain works, and seeing the difference between alexmac's promise.all vs your sequential was educational. – Trees4theForest Aug 11 '17 at 23:19
@NoahFreitas in your current implementation, the files still loaded in sequence. `Promise.all` just accepts the list of values, not promises, and returns them immediately, so it's not needed there. – alexmac Aug 11 '17 at 23:23
1

@alexmac the function that I am mapping over the `files` array with is returning a promise (because it's `async`). Thus at the end of evaluating `files.map()`, I have an array of promises. I use `Promise.all` to wait for them all to resolve. The mapping function is running over each file name sequentially but it is not waiting for `slow_promise_function` to return before processing the next. – Noah Freitas Aug 11 '17 at 23:26

score 1 · Answer 3 · answered Aug 11 '17 at 22:58

Well that can simply be done with Promises more info on this link Promises.

const newStuffFunc = async () => {
 try {
   let newStuff = await slow_promise_function(f);

   // new stuff is a promise. so you either do it with await again or with then/catch

   let data = await newStuff;
 } catch (e){
 }

}

const slow_promise_function = (url) => {
  return new Promise((resolve, reject) => {
     // do something asynchronous which eventually calls either:
    //
    //   resolve(someValue); // fulfilled
    // or
    //   reject("failure reason"); // rejected
  });
};

This link can show you more usability of async/promises into javascript.

Javascript implement async to get result of long-running process into array

3 Answers3