Puppeteer evaluate returns undefined for an array of DOM nodes

Question

I have a table on a website and I want to scrape all tds with class='col-sm-1 available-day'.

Using Puppeteer I tried like this:

 let result = await page.evaluate(() => {

     return Array.from(document.querySelectorAll('td.col-sm-1.available-day'));
});

But when I try to output the result it says that is undefined. How can I return an array of DOM nodes from evaluate?

Can you show the full undefined error? What exactly is undefined? — Nathan, Oct 05 '20 at 22:28
There is no error , when I type console.log(result) it says just : undefined — Marco, Oct 05 '20 at 22:32
You can't return elements with evaluate, try evaluateHandle or otherwise return just text — pguardiario, Oct 06 '20 at 02:05
There's also `page.$$` if you want an array of ElementHandles. — ggorlen, Dec 22 '22 at 01:51

score 3 · Accepted Answer · answered Oct 06 '20 at 03:11

3

This is expected behaviour. document.querySelectorAll gets DOM nodes which are complex object that cannot be returned from page.evaluate.

You need to return simple objects that can be serialized, for example, here we are returning an array of text values for the selected elements:

let result = await page.evaluate(() => {
  const tds = Array.from(document.querySelectorAll('td.col-sm-1.available-day'));
  return tds.map((td) => td.innerText);
});

answered Oct 06 '20 at 03:11

Vaviloff

16,282
6
48
56

Thank you , works <3 . I have one more question : so now the result its defined but has 0 values in it. When I go to my url from which I want to scrape some data ( https://www.drpciv.ro/ ) and I try to take a screenshot it shows blank. Any idea why I am not able to reach that url with puppeteer ? – Marco Oct 06 '20 at 07:29
1

Maybe they don't want puppeteer to access their site (in that case research how to change user agent). – Vaviloff Oct 06 '20 at 08:17

Puppeteer evaluate returns undefined for an array of DOM nodes

1 Answers1

Linked