Puppeteer Not Waiting Until Page Load Complete

Question

I wrote a script to take screenshots of 50 size charts on a webpage. Each chart is contained in a element. The funny thing is, only the first three table charts are captured, the rest of the PNG files are blank, completely white.

Since the charts are pulled from a database, I thought it could be that the page hadn't finished loading before the screenshots are taken, so I added { "waitUntil": "networkidle0" } but that didn't solve anything. Still, the script only creates screenshots of the first three charts, 0.png, 1.png and 2.png. The rest of the PNG files, 3.png - 49.png, are created but just white data.

What could be the issue? If I visit the page on my browser, all 50 charts load perfectly, so why is Puppeteer making a screenshot of only the first three? Here is my script:

const puppeteer = require( 'puppeteer' );

( async() => {
    const browser = await puppeteer.launch();

    const page = await browser.newPage();
    await page.goto( 'http://www.example.com/size-charts.php', { "waitUntil": "networkidle0" } );

    // Get a list of all elements.
    const elements = await page.$$( 'div.chartContainer' );

    for( let i = 0; i < elements.length; i++ ) {
        try {
            // get screenshot of a particular element
            await elements[ i ].screenshot( { path: `${ i }.png` } );
        }
        catch( e ) {
            console.log( `Couldn't take a screenshot of the element with the index of: ${ i }. Reason: `, e );
        }
    }

    await browser.close();
} )();

Seems likely some rendering is going on after the network finishes. So, should wait for something else. Hard to say what, since no chart source given, but here are some ideas [puppeteer : wait an element is visible](https://stackoverflow.com/a/54103671/4716245) — Richard Matsen, May 31 '19 at 19:33
I am very new to Puppeteer, can you show me an example using my code? — GTS Joe, May 31 '19 at 20:42

score 3 · Accepted Answer · answered Jun 04 '19 at 14:43

I'm going to answer my own question in the hopes it can help others. In my particular case, the solution was setting the viewport height to a very large number, e.g.:

page.setViewport( { width: 1920, height: 100000 } );

After that, the script was able to create screenshots of all the selected elements.

score 0 · Answer 2 · answered Jun 02 '19 at 09:12

While the actual solution can/should be waiting for each element to be visible, there are some possible other solutions available.

Possible solution 1

You can scroll to the element, wait for some moment to render properly, and then take screenshot.

elementHandle.screenshot() scrolls to the element, but does not have a way to delay or wait for element to be visible.

A quick search on official repo showed there are at least 19 open issues mentioning blank screenshot at this momement.

Instead, we can use a custom .evaluate, or a .hover etc. to scroll to the element before taking screenshot.

elementHandle.hover(): This method scrolls element into view if needed, and then uses page.mouse to hover over the center of the element. If the element is detached from DOM, the method throws an error.

Let's use it,

// Get a list of all elements.
const elements = await page.$$('div.chartContainer');

for (let i = 0; i < elements.length; i++) {
  // scrolls into view and hovers the element
  await elements[i].hover();

  // wait for some random number 
  await page.waitFor(1000);

  // get screenshot of a particular element
  await elements[i].screenshot({
    path: `${ i }.png`
  });
}

Possible solution 2

Some reported that using elementHandle.boundingBox() fixed their issue. What this does is, grabs the position, height, width etc for the element and uses that for screenshot.

// Get a list of all elements.
const elements = await page.$$('div.chartContainer');

for (let i = 0; i < elements.length; i++) {
  // get screenshot of a particular area
  await page.screenshot({ // <-- use page here
    path: `${ i }.png`,
    clip: await elements[i].boundingBox() // <-- use clip here
  });
}

Puppeteer Not Waiting Until Page Load Complete

2 Answers2

Possible solution 1

Possible solution 2