1

If we are unaware of how many list items are fetched, how to choose selectors and wait for all list items being loaded. Lets imagine they are fetched through ajax.

Lets say following is the vague html fetched and at max 5 li elements. We are not already aware of how many exist on a particular fetch. But maybe in the real scenario there could be 100 list items.

    <ul class='container'>
        <li>item 1</li>
        <li>item 2</li>
        <li>item 3</li>
        <li>item 4</li>
        <li>item 5</li>
    </ul>

What should be our puppeteer wait api or whatever thats appropriate which is the simplest way to fetch without missing anything.

Do methods like following work ? (but it still seems vague, and mostly results in navigation or some other errors)

    await page.evaluate(() => 
          document.querySelector('.container').childNodes.length>3
    );
Shyam R
  • 473
  • 1
  • 5
  • 17

1 Answers1

0

You can use .waitForFunction for waiting until the condition is true.

In this case: wait until .container has 3 children minimal.

await page.waitForFunction(() => {
  return document.querySelector('.container').children.length > 3;
});

Simpler way:

await page.waitForFunction(() => {
  return document.querySelector('.container').childElementCount > 3;
});
Jordy
  • 1,802
  • 2
  • 6
  • 25
  • but this still doesnt work.. sometimes lets say the list has 100 items.. so if the processing starts immediately after 3rd item, then there are some network issues encountered.. is there any other way that works whatever the items length is still waits until all content are loaded – Shyam R Apr 22 '23 at 07:39
  • @ShyamR Have you tried to use `.setInterval`? – Jordy Apr 22 '23 at 10:23
  • @ShyamR if you don't know how long it'll take to load all of the items, and you don't know how many items to wait for, then there's nothing to do but pick an arbitrary timeout or keep polling for some arbitrary period. Almost no actual sites are this flakey, so if you don't mind sharing the site and more context, there's almost certainly a way to add precision to your specification. Maybe see [this answer](https://stackoverflow.com/a/72535642/6243352) for more details, especially the bottom section. Maybe monitoring network requests would help? – ggorlen Apr 22 '23 at 18:37