Promises in Promise.all() return undefined sometimes / crawler-request package nodejs

Question

Im not sure if im using promise.all wrong, or if the nodejs package I use for retrieving pdf remotly and parsing these is overwhelmed by too many requests at once.

https://codesandbox.io/s/sharp-wave-qikvb // here the codesandbox

I attempted to use promise.all

let urls = [arrayofURLS];

function pdfData() {   
    return Promise.all(
          urls.map(item => {
            this.crawlPdf(item);
          })
        )
          .then(result => {
        // handle result
          })

}

This is the function that uses the crawler package (called crawler-request) :

    crawlPdf: async function(Url) {

      return new Promise(async function(resolve, reject) {
      let response = await crawler(Url);
      resolve(response.text);
   }

2 out of 5 requests are usually undefined. But sometimes everything works o.O..

That shouldn't work - you're always mapping to an array of `undefined`s. Return the created Promise instead — CertainPerformance, Jun 21 '19 at 02:22
change "sometimes" in the title to "always" as your map function always returns undefined — Jaromanda X, Jun 21 '19 at 02:25
[Never pass an `async function` as the executor to `new Promise`](https://stackoverflow.com/q/43036229/1048572)! Actually [this version](https://stackoverflow.com/revisions/56696074/3) of your `crawlPdf` code was fine. — Bergi, Jun 21 '19 at 15:11

score 1 · Answer 1 · answered Jun 21 '19 at 02:28

1

You must return promises to the all method. Right now you aren't returning anything so it looks like Promise.all([undefined, undefined, undefined])

Since it looks like you can use arrow functions, you can just switch your curly brackets for parens, or put it on one line and get rid of the brackets completely - these formats always return the result of the function body.

urls.map(item => (
  this.crawlPdf(item)
));

urls.map(item => this.crawlPdf(item));

Or keep it explicit

urls.map(item => { return this.crawlPdf(item) });

answered Jun 21 '19 at 02:28

jhummel

1,724
14
20

im returning the response , forgot to add that. .. in crawlPdf function – balui Jun 21 '19 at 02:35
Do you mean you are returning the data from that function? You need to return an actual promise. `Promise.all` accepts an array of promises, not an array of data. the `crawlPdf` function should look something like: `function () { return new Promise(...) }` – jhummel Jun 21 '19 at 02:41
Adjusted the function ... this is how it actually looks – balui Jun 21 '19 at 02:47
That looks great, but then you have to return that promise from your map callback like I explained above. You need both returns – jhummel Jun 21 '19 at 02:56
i dont understand ... my original code already returning the promise if the response = await crawler(Url); actually gives a response ... – balui Jun 21 '19 at 03:28
ok I replicated the code on codesandbox , this should be clear now .. hope you can help https://codesandbox.io/s/sharp-wave-qikvb – balui Jun 21 '19 at 14:26
sometimes im getting this error (node:369) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'length' of null – balui Jun 21 '19 at 14:33
idk maybe my internet connection was lagging or smth ... but on my machine i got the error plenty of times – balui Jun 21 '19 at 14:53

Promises in Promise.all() return undefined sometimes / crawler-request package nodejs

1 Answers1