Puppeteer - Passing a variable to page.evaluate()

Question

I'm trying to scrape a bunch of tables on a page with Puppeteer but the amount of tables varies each time and therefore i need to pass a variable into the page.evaluate() function.

I have an array containing the variables i need to use but lets say i wanna check the second table manually using a variable, here's how i approached the problem initially:

const x = 2
let value = await page.evaluate(() =>
document.querySelector("#pageBody > div:nth-child(" + x + ") > table > tr > td").innerText
);
console.log(value);

//Evaluation failed: ReferenceError: x is not defined

After some research,from what i understand, the correct way of passing a variable in there is:

const x = 2;
let value = await page.evaluate((x) => { 
document.querySelector("#pageBody > div:nth-child(" + x + ") > table > tr > td").innerText 
}, x);
console.log(value);

//value = undefined

But since im not getting the results i want, there's something im doing wrong or not understanding properly. Any ideas?

score 3 · Accepted Answer · answered Nov 05 '19 at 21:58

3

Figured it out. Of course 5 minutes after posting to SO following hours of frustration, but here goes:

const x = 2;
let value = await page.evaluate((x) => { 
return Promise.resolve(document.querySelector("#pageBody > div:nth-child(" + x + ") > table > tr > td").innerText) 
}, x);
console.log(value);

answered Nov 05 '19 at 21:58

Burgese

43
5

1

You don't need Promise.resolve for that FTR, you just needed the `return` – pguardiario Nov 06 '19 at 00:33
I did do a try with that earlier but i didnt get it to work. Maybe messed something else up in those tries. Could my way of doing it with ```Promise.resolve()``` cause any problems? (fairly new to async js) – Burgese Nov 06 '19 at 15:30

score 1 · Answer 2 · answered Nov 06 '19 at 00:39

1

I'll just add this to settle some confusion, if you do it in a one line arrow function you don't need the return:

const css = "#pageBody > div:nth-child(" + x + ") > table > tr > td"
let value = await page.evaluate((css) => document.querySelector(css).innerText), css);

answered Nov 06 '19 at 00:39

pguardiario

53,827
19
119
159

1

Part from that you had one too many closing brackets that worked well aswell. Correct syntax would be: ```let value = await page.evaluate((css) => document.querySelector(css).innerText, css);``` – Burgese Nov 06 '19 at 14:28

Puppeteer - Passing a variable to page.evaluate()

2 Answers2