2

I am using puppeteer to perform some action and to take screen shots. The entire process has multiple authentication, first Gmail login, then the SSO redirects to Microsoft and from them we get into the application. The problem that i am facing is, when i set my browser - headless : false, everything works fine, but when i set my headless: true, the Gmail started asking for captcha and some time it says, the browser is not safe.

I have been trying to sort this out for quite some time and i am completely blinded. I have read throug the internet and found lot of options like seting up useragent, adding preference etc, but non of them are working.

Here is my simple code and with the screen shots.

// puppeteer-extra is a drop-in replacement for puppeteer,
// it augments the installed puppeteer with plugin functionality
const puppeteer = require('puppeteer-extra');
//puppeteer.use(require('puppeteer-extra-plugin-stealth')());

// add stealth plugin and use defaults (all evasion techniques)
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
console.log(StealthPlugin.availableEvasions);
puppeteer.use(StealthPlugin());
//const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha')


//const puppeteer = require('puppeteer')

// puppeteer usage as normal
puppeteer.launch({ headless: true, executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', setUserAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36' }).then(async browser => {
  console.log('Running tests..')
  const page = await browser.newPage()
  //await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36')
  await page.goto('https://accounts.google.com/signin/v2/identifier?service=mail&passive=1209600&osid=1&continue=https%3A%2F%2Fmail.google.com%2Fmail%2Fu%2F0%2F&followup=https%3A%2F%2Fmail.google.com%2Fmail%2Fu%2F0%2F&emr=1&flowName=GlifWebSignIn&flowEntry=ServiceLogin')
  await page.waitForTimeout(5000)
  await page.setViewport({ width: 1300, height: 950 }); 
  await page.waitForTimeout(1000)
  await page.screenshot({path: '1.png', fullPage: true})
  await page.waitForSelector('[type="email"]')
  await page.type('[type="email"]', 'nxxxx@xxxx.com');
  await page.keyboard.press('Enter');

  await page.waitForTimeout(1000);
  //await page.solveRecaptchas()
  //await page.waitForNavigation({'waitUntil':'domcontentloaded'});
  await page.screenshot({path: '2.png', fullPage: true})
  await page.screenshot({ path: 'testresult.png', fullPage: true })
  await browser.close()
  console.log(`All done, check the screenshot. ✨`)
})

I have updated only a part of my code, but still, the result is the same, unless i bypass the detection , i am unable to move forward. please find the screen shot attached.

Output of the Image

All i wanted is to b pass this stage and go to the next screen.....

Please help.... :(

ggorlen
  • 44,755
  • 7
  • 76
  • 106
Sathya Narayanan
  • 23
  • 4
  • 1
    well, this is not how you should do it. If Google doesn't want you to use an automation tool to log in accept it. And, even though you bypassed it today, there is a high chance that Google might add new pages while registering in the future in name of security. Playing cat & mouse game is futile here :). – Shirshak55 May 30 '22 at 16:00
  • Also, it it works for headful why do you even need headless? You can use tools like xfvb to run in ci. – Shirshak55 May 30 '22 at 16:01
  • Hi, Thanks for your response yes, but the entire puppeteer is running inside a docker container, and i cant use xfvb as well.... – Sathya Narayanan May 30 '22 at 16:10
  • There is no good solution then. Google sometimes suspects due to different variables like IP, fingerprinted data, etc... Headless and headful modes have different subtle differences. For instance --automation flags, etc are passed, navigators' properties are changed, etc.. And sometimes Google shows verification screens like phone number confirmations etc.. Hence, I think you are approaching the problem in the wrong way. – Shirshak55 May 30 '22 at 17:46
  • Try firefox too. – Shirshak55 May 30 '22 at 17:47
  • Any help on this request ? – Sathya Narayanan Jun 10 '22 at 08:26
  • Does this answer your question? [Why does headless need to be false for Puppeteer to work?](https://stackoverflow.com/questions/63818869/why-does-headless-need-to-be-false-for-puppeteer-to-work) – ggorlen Jul 17 '22 at 14:42

1 Answers1

6

If it works in headful mode, but not in headless mode, a solution would be to use Xvfb. So use headless: false in the launch options, and run the script with xvfb-run.

Xvfb is an X server that can run on machines with no display hardware and no physical input devices, like the container you are working with. Xvfb emulates a dumb framebuffer using virtual memory.

Chrome (or Firefox for that matter) renders its graphics in a window created by an X client library like XCB. XVFB opens that window (on your container).

In Ubuntu you can install Xvfb with:

sudo apt-get install xvfb

The following command will run your Puppeteer script and render Chrome in a virtual framebuffer:

xvfb-run --auto-servernum node your-puppeteer-script.js

Keep in mind that Puppeteer itself uses xvfb for its CI tests.

jackdbd
  • 4,583
  • 3
  • 26
  • 36