page.$eval() selector only using visible element

Question

I'm working with puppeteer at the moment to create a web-crawler and face the following problem:

The site I'm trying to scrape information off of uses Tabs. It renders all of them at once and sets the display-property of all but one tab to 'none' so only one tab is visible.

The following code always gets me the first flight row, which can be hidden depending on the date that the crawler is asking for.

const flightData = await page.$eval('.available-flights .available-flight.row', (elements) => {
  // code to handle rows
}

There doesn't seem to be an additional parameter you can pass with .$eval() like you can in

.waitForSelector('.selector', {hidden: false})

Am I following the wrong idea? Is there a way to only select the shown element and work with that data?

https://stackoverflow.com/a/54103671/10761855 – Vadim Hulevich Jul 14 '20 at 08:57 — Vadim Hulevich, Jul 14 '20 at 08:57

score 1 · Accepted Answer · answered Jul 14 '20 at 08:47

1

const flightData = await page.$eval('.available-flights .available-flight.row:not([style*="display:none"]):not([style*="display: none"])', (elements) => {
  // code to handle rows
}

Does the trick :)

answered Jul 14 '20 at 08:47

Pascal Stockert

356
1
12

page.$eval() selector only using visible element

1 Answers1