How to change Promise.all to send requests one by one

Question

I have a an array of chunked data that I need to upload one chunk at time. The current implementation I used it to encapsulate the logic in an Promise.all() since I need to return the result of the promise, The problem with this approach is that all the upload is done asynchronously resulting in a Timeout error as the server can't process all the requests at the same time, How can I modify this method so that the upload is done one chunk at time ?.

My code:

var chunks = _.chunk(variableRecords, 30);
return Promise.all(
        chunks.map(chunk => this.portalService.updateDataForChart(variableId, chunk)))
        .then((updateRes: boolean[]) => {
          if (updateRes.every(updateStatus => updateStatus)) {
            return this.executeRequest<HealthDataSource, boolean>({
              path: `/variable/user/datasources/${dataSource.identifier}`,
              method: 'PUT',
              body: {
                libelle: dataSource.datasource.libelle,
                type: dataSource.datasource.type,
                lastSyncDate: Math.max(maxDate, dataSource.datasource.lastSyncDate)
              },
              headers: this.getHeaders()
            });
          } else {
            return false;
          }
        });

Possible duplicate of [Resolve promises one after another (i.e. in sequence)?](https://stackoverflow.com/questions/24586110/resolve-promises-one-after-another-i-e-in-sequence) — ponury-kostek, Nov 29 '19 at 14:40

Renaldo Balaj · Accepted Answer · 2019-11-29T14:15:55.580

5

You need them in SEQUENCE , for of is the way to go :

async function chunksSequence(chunks) {
  for(const chunk of chunks) {
    await // your other code here
  }
};

If you need to return something

async function chunksSequence(chunks) {
  let results = []
  for(const chunk of chunks) {
    let result = await // your other code here
    results.push(result)
  }
  return results 
};

Because of comment needed in a promise on return

async function chunksSequence(chunks) {
 return new Promise((resolve, reject)=>{
    let results = []
    for(const chunk of chunks) {
      let result = await // your other code here
      results.push(result)
    }
    resolve(results) 
  }
};

edited Nov 29 '19 at 14:15

answered Nov 29 '19 at 13:40

Renaldo Balaj

2,390
11
23

How do you get all the results at the end ? – Weedoze Nov 29 '19 at 13:49
1

@Weedoze, simply add a collector array before the loop, push every result to it and return that at the end of the function. – CerebralFart Nov 29 '19 at 13:51
1

@CerebralFart Indeed, it would be better to add this part to the code as OP needs to handle the final results – Weedoze Nov 29 '19 at 13:51
1

done eddited, with the return part, give feedback if needed :) – Renaldo Balaj Nov 29 '19 at 13:52
the return type should be a Promise as I'm going to use a .then on it to execute the PUT request – R0b0t0 Nov 29 '19 at 14:13
done added your case too :) check the eddited answer, feel free to have it as the accepted answer hehehe – Renaldo Balaj Nov 29 '19 at 14:16

score 2 · Answer 2 · answered Nov 29 '19 at 13:40

2

You can do this with the help of Array.reduce()

const chunks = _.chunk(variableRecords, 30);
return tasks.reduce((promiseChain, currentTask) => {
    return promiseChain.then(chainResults =>
        currentTask.then(currentResult =>
            [ ...chainResults, currentResult ]
        )
    );
}, Promise.resolve([])).then(arrayOfResults => {
    // Do something with all results
});

Source : https://decembersoft.com/posts/promises-in-serial-with-array-reduce/

answered Nov 29 '19 at 13:40

Weedoze

13,683
1
33
63

This assumes that `variableRecords` is an array of promises, wherein the process of requesting from the server has already been established. It will still have the same problem as the OP, as the server can't process all requests at the same time. – ryeballar Nov 30 '19 at 00:21

score 0 · Answer 3 · answered Nov 29 '19 at 14:05

If you don't / can't use await you could use something like this

function runSequenceItem(chunks, index) {
 return new Promise(chunks[index])
  .then(res => {
     index ++
     if (index < chunks.length) {
       return runSequence(chunks[index], index + 1)
     } else {
       // this is not needed actually
       return 'done'
     }
   })
}

function runInSequence(chunks) {
 return runSequenceItem(chunks, 0)
}

If you also need the results then you can return an array at the end of the recursion runInSequence

    function runSequenceItem(chunks, index, results) {
 return new Promise(chunks[index])
  .then(res => {
     results.push(res)
     index ++
     if (index < chunks.length) {
       return runSequence(chunks[index], index + 1)
     } else {
       return results
     }
   })
}

function runInSequence(chunks) {
 return runSequenceItem(chunks, 0, [])
}

and then retrieve it at the end

let results = runInSequence(chunks)

How to change Promise.all to send requests one by one

3 Answers3