Puppeteer can't catch failing request & errors

Question

I trying to collect data from failing requests and js error.

I'm using the following site: https://nitzani1.wixsite.com/marketing-automation/3rd-page

The site has a request to https://api.fixer.io/1latest, which returns a status code of 404,

also the page contains thw following js error:

"Uncaught (in promise) Fetch did not succeed"

I've tried to code bellow to catch the 404 and js error but couldn't. Not sure what I'm doing wrong, any idea as to how to solve it?

const puppeteer = require('puppeteer');

function wait (ms) {
    return new Promise(resolve => setTimeout(() => resolve(), ms));
}

var  run = async () => {
    const browser = await puppeteer.launch({
        headless: false,
        args: ['--start-fullscreen']
    });

    page = await browser.newPage();

    page.on('error', err=> {
        console.log('err: '+err);
    });

    page.on('pageerror', pageerr=> {
        console.log('pageerr: '+pageerr);
    });

    page.on('requestfailed', err => console.log('requestfailed: '+err));

    collectResponse = [];

    await page.on('requestfailed', rf => {
        console.log('rf: '+rf);
    });

    await page.on('response', response => {
        const url = response.url();
        response.buffer().then(
            b => {
                // console.log(url+' : '+response.status())
            },
            e => {
                console.log('response err');
            }
        );
    });
    await wait(500);

    await page.setViewport({ width: 1920, height: 1080 });
    await page.goto('https://nitzani1.wixsite.com/marketing-automation/3rd-page', {
    });
};

run();

The target page itself shows me fetch error even without puppeteer. What are you expecting and what are you getting? — Md. Abu Taher, May 25 '18 at 19:51
Have you found the solution? tried `page.on('console')` with `error` type doesn't work too. — otong, Mar 01 '19 at 08:46

score 12 · Answer 1 · answered Dec 07 '19 at 07:16

The complete worked answer is:

const puppeteer = require('puppeteer');

const run = async () => {
    const browser = await puppeteer.launch({
        headless: true
    });

    const page = await browser.newPage();
    // Catch all failed requests like 4xx..5xx status codes
    page.on('requestfailed', request => {
        console.log(`url: ${request.url()}, errText: ${request.failure().errorText}, method: ${request.method()}`)
    });
    // Catch console log errors
    page.on("pageerror", err => {
        console.log(`Page error: ${err.toString()}`);
    });
    // Catch all console messages
    page.on('console', msg => {
        console.log('Logger:', msg.type());
        console.log('Logger:', msg.text());
        console.log('Logger:', msg.location());

    });

    await page.setViewport({ width: 1920, height: 1080 });
    await page.goto('https://nitzani1.wixsite.com/marketing-automation/3rd-page', { waitUntil: 'domcontentloaded' });
    await page.waitFor(10000); // To be sure all exceptions logged and handled
    await browser.close();
};

run();

Save in .js file and easily run it.

Note that as of January 2022 `page.waitFor()` is deprecated and the same functionality can be achieved with `page.waitForTimeout(10000);` — Austin R. Scott, Jan 14 '22 at 19:11
`requestfailed` will NOT catch all failed requests such as 4xx..5xx status codes. Quote from puppeteer documentation at https://pptr.dev/api/puppeteer.httprequest - `HTTP Error responses, such as 404 or 503, are still successful responses from HTTP standpoint, so request will complete with requestfinished event.` — mohonish, Jun 28 '23 at 00:04

score -1 · Answer 2 · answered Mar 25 '21 at 14:57

Current puppeteer 8.0.0^ have a very small amount of information in message.text(). So we need to get a description of the error from JSHandle. Please check this comment with fully descriptive console errors from JSHandle object

Check the link here https://stackoverflow.com/a/66801550/9026103

Puppeteer can't catch failing request & errors

2 Answers2

Linked