1
const puppeteer = require('puppeteer');
(async () => {
    const browser = await puppeteer.launch({args: ['--no-sandbox']});
    const page = await browser.newPage();

    try {
        await page.goto('https://www.allabout*****.org', {waitUntil: 'networkidle2'}); // 59 second - load - domcontentloaded - networkidle2
        const cookies = await page._client.send('Network.getAllCookies');
        JSON.stringify(cookies, null, 4);

    } catch (e) {

    }
    await browser.close();
})();

Tried above code to fetch all cookies but it only takes cookies before accepting captcha. After accepting captcha also, it sets cookies, that are not getting in the cookie list. How can I escape captcha using puppeteer in headless mode ( headless: true)?

Tried different independent node modules, puppeteer helpers, nothing worked.

mujuonly
  • 11,370
  • 5
  • 45
  • 75

1 Answers1

1

I just tried your code and realized that you used {waitUntil: "load"}. I just replaced that with {waitUntil: "networkidle2"} and removed the timeout(still used headless:true) and it showed me the CookieScriptConsent on cookie list.

Naimur Rahman
  • 697
  • 4
  • 15
  • Please see the [screenshot](https://i.ibb.co/GCtDZwx/Screenshot-2019-10-10-at-9-10-41-AM.png), it's not working – mujuonly Oct 10 '19 at 03:42
  • 1
    I again tried on try puppeteer. Check out this screenshot https://i.imgur.com/ARLYuu2.png – Naimur Rahman Oct 10 '19 at 06:32
  • But it's not working in my demo server. (exact same code ). Those cookies are getting set after the captcha is verified. So there may be some whitelisting needed from the Cloudflare side, as this is a Cloudflare hosted website. Thanks though – mujuonly Oct 10 '19 at 10:16
  • configured try-puppeteer in localhost - then it's not working ( not bypassing captcha) – mujuonly Oct 22 '19 at 06:13
  • If you do not bypass captcha then you can't get the "CookieScriptConsent" I think. – Naimur Rahman Oct 23 '19 at 07:14
  • I cloned try-puppeteer from GitHub and run the script, then the cookie is not listing. Lost – mujuonly Oct 23 '19 at 07:17
  • @md-abu-taher CAn you have a look into this question – mujuonly Oct 24 '19 at 11:32