Puppeteer losing context before retrieving lis

Question

I am trying to use node/puppeteer to retrieve items from an unordered list. I am able to navigate to the page, perform a search, but when I attempt to generate an array with the lis, it breaks with the following error:

UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Execution context was destroyed.

Responding to the comment, here is the full code:

require('dotenv').config();
const puppeteer = require('puppeteer');
const ac = require("@antiadmin/anticaptchaofficial");

(async () => {

    ac.setAPIKey(process.env.ANTICAPTCHA_KEY);
    ac.getBalance()
        .then(balance => console.log('my balance is $'+balance))
        .catch(error => console.log('received error '+error))

    console.log('solve recaptcha first');
    let token = await ac.solveRecaptchaV2Proxyless('https://secure.meetup.com/login','6LcA8EUUAAAAAG17qfEfNaX6H8ozmI-IvmokZUnZ');
    if (!token) {
        console.log('something went wrong with captcha solving');
        return;
    } else {
        console.log('token is: ', token);
    }
    console.log ('opening browser');
    const browser = await puppeteer.launch({
        headless: false
    });

    console.log('creating new tab');
    const page = await browser.newPage();

    console.log('setting page size');
    await page.setViewport({width: 1368, height: 1080})

    console.log('opening target page');
    await page.goto('https://secure.thesite.com/login', {waitUntil: 'networkidle2'});

    await page.type('#email', process.env.MY_EMAIL)
    await page.type('#password', process.env.MY_PASSWORD)

    console.log('click login button');
    await page.evaluate((token) => {
        const textarea = document.querySelector("textarea#g-recaptcha-response.g-recaptcha-response")
        if (textarea) {
            textarea.innerText=token
        }
        const button = document.querySelector("#loginFormSubmit")
        button.disabled = false
        button.click()
    }, token)

    console.log('Entering keywords');
    await page.waitForSelector("input#mainKeywords.dropdown-toggle.ellipsize")
    await page.type("input#mainKeywords.dropdown-toggle.ellipsize","write")

    console.log('Click to search and filter fr groups')
    await page.evaluate(async () => {
        const searchForm = document.querySelector("#searchForm")
        searchForm.submit()
        const groupButton = document.querySelector("#simple-view-selector-group")
        await groupButton.click()
        function sleep(seconds) {
            return new Promise((resolve) => {
                setTimeout(() => resolve(true), seconds*1000)
            });
        }
        await sleep(2);
    })
    console.log('start grroups check')
    const groups = page.evaluate(async () => {
        console.log('setting grroups aray');
        const groups=[];
        console.log('about to awaait selector');
        if (await page.waitForSelector('a.groupCard--photo.loading.nametag-photo') !== null) {
            const groupList = document.querySelectorAll("a.groupCard--photo.loading.nametag-photo")
            console.log("groupis is: ", groupList);
            groupList.forEach(group => {
                groups.push(group.href);
            })
            return groups;
        } else {
            console.log('selector is null');
        }

    })
    console.log(groups);

})();

score 0 · Answer 1 · answered Jul 18 '21 at 12:05

Your puppeteer script is a bit overcomplicated. I am unable to reproduce an Execution context was destroyed scenario (of course with a different page, as I don't have a Meetup account), but the groups function expression is not valid code for sure, that causes most of your issues:

you need to await the page.evaluate method as it returns a promise, it will misbehave otherwise (e.g. logs [object Promise] only etc.)
you cannot use puppeteer API methods within the page context (inside a page.evaluate):

if (await page.waitForSelector('a.groupCard--photo.loading.nametag-photo') !== null)

You rather should move this condition outside of evaluate.

Returning an array from page.evaluate is trickier than you'd expect (see existing questions and answers on this).

Suggestion

I'd refactor this part to avoid the above issues and avoiding page context issues too. You can use page.$$eval:

const groups = await page.$$eval('a.groupCard--photo.loading.nametag-photo', (groupList) => 
  groupList.map((group) => group.href)
)

Puppeteer losing context before retrieving lis

1 Answers1

Suggestion