Puppeteer identifier string variable won't parse; Unsure why

Question

I'm trying to have a string be used inside a puppeteer string, it won't work for some reason.

Specifically with this code

await page.waitForSelector('div[class = "sh-dlr__list-result"')

When i try to parse in a variable

let identified1 = 'div[class = "sh-dlr__list-result"'

so making

await page.waitForSelector(identified1)

It won't work. This is really limiting, is there a way around this issue?

This is the expanded code

https://jsfiddle.net/hewlbern/6p7kdozt/10/ Run it in your computer, jsfiddle unsure if I can run it from there.

I believe it is creating a cors error now - very weird! Why would using a variable create a cors error : /

Thanks!

That's impossible, could you share the rest of your code and where exactly are you declaring that variable so we can see what is happening? You can also just for sanity check log the value of `identified1 ` just before `await page.waitForSelector(identified1)` — Luís Ramalho, Apr 20 '20 at 05:33
@LuísRamalho Yeah would appreciate your help on this one :) Can you see the jsfiddle? — LeCoda, Apr 20 '20 at 05:41
Since your selector is missing the closing `]` in both cases, I'd be very surprised if either worked — Phil, Apr 20 '20 at 05:44
the closing ] isn't needed actually, but i can close it regardless. the console is calling the string. I believe it's a cors issue now? — LeCoda, Apr 20 '20 at 05:47
But that doesn't make any sense - why would making it a const bring about a cors issue — LeCoda, Apr 20 '20 at 05:48
I've ran your code (https://gist.github.com/luisramalho/c04a4b656a44acf65c298e2ac5d553c1) on https://try-puppeteer.appspot.com/ and it worked fine :) -- got an array of `{"Product":"Premium Nature Pure Aloe Vera Gel 12 oz 4 oz","Price":"$7.99."}` — Luís Ramalho, Apr 20 '20 at 07:05
Query blue light glasses, it has separate identifiers. When I use a if statement for changing the identifiers it breaks with cors error — LeCoda, Apr 20 '20 at 07:11
@LuísRamalho did you manage to see if the other query worked? — LeCoda, Apr 20 '20 at 08:44
I'm not sure what you mean @MichaelHolborn, do you ming creating a gist that I can try on try-puppeteer.appspot.com? Thank you! — Luís Ramalho, Apr 20 '20 at 08:50
Hey @MichaelHolborn, I think I found the cause of the issue and you were right. It didn't work because the code was inside the `page.evaluate()`. I left an answer explaining my findings, I hope you can test it out. It seems to work now in https://try-puppeteer.appspot.com/ — Luís Ramalho, Apr 20 '20 at 11:24

score 2 · Accepted Answer · answered Apr 20 '20 at 11:22

2

The reason is because you're declaring identified inside the page.evaluate(). So, when you do the following it's already out of scope.

if (currentPage < pagesToScrape) {
  console.log(identified1);
  await Promise.all([
    await page.click(buttonSelector),
    await page.waitForSelector(identified),
  ]);
}

You did log the identified1 but you're using identified for the selector.

You'll have to pass the identifier2 to the pageFunction like so:

let newProducts = await page.evaluate(({ identifier2 }) => {
  // ...
},{ identifier2 });

See here some examples:

answered Apr 20 '20 at 11:22

Luís Ramalho

10,018
4
52
67

Awesome. If i wanted to do the logic operator to get the correct identifier, regardless of query (so it works it out itself), how would I test it without it being inside a page.evaluate? – LeCoda Apr 20 '20 at 12:38
Updated the example again to better explain. Essentially, I want to allow for different types of selectors, and be able to parse in the correct selector. :) – LeCoda Apr 20 '20 at 13:05
1

It's hard to help you further, the jsfiddle is not working and I'm not sure I fully understand what you're trying to do. I think the key takeaway is that if you want to use a variable inside the `page.evaluate()` you'll have to pass it to the `pageFunction`. See if [this](https://stackoverflow.com/questions/46088351/puppeteer-pass-variable-in-evaluate) is helpful. – Luís Ramalho Apr 21 '20 at 00:42
1

thanks for answering. If possible, take down the gist. Cheers! – LeCoda Apr 21 '20 at 08:43

Puppeteer identifier string variable won't parse; Unsure why

1 Answers1