1

I have a script that should open a headless browser, make a search, and take a screenshot.

When I'm running the script with headless: false, everything works.

When I'm running it with headless: true, I get blocked by the website (captcha).

I know that there are ways to bypass captchas but that's not what I want.

My question is what's the difference between these 2 modes that the website recognize me as headless browser but when it's not headless it doesn't.

Things I've tried:

  • pass chrome check
  • pass notification check
  • set language header
  • insert new user agent and other different args, such as:
'--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36',
'--window-size=1920,1080',
'--no-sandbox',
'--disable-gpu',
'--no-zygote',
'--disable-setuid-sandbox',
'--disable-accelerated-2d-canvas',
'--disable-dev-shm-usage'

I'm using latest puppeteer version(10.1.0), 'puppeteer-extra-plugin-stealth' and 'puppeteer-extra'.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
nadavshkoori
  • 11
  • 2
  • 2
    Does this answer your question? [Headless browser detection](https://stackoverflow.com/questions/55364643/headless-browser-detection) – theDavidBarton Aug 15 '21 at 12:50
  • Welcome to SO! Adding a [mcve] on the site in question might help to give you a more concrete reason as to what's going on. It's always best to have something to run and fiddle with. – ggorlen Aug 15 '21 at 14:00
  • @theDavidBarton tried all of these and still doesn't work – nadavshkoori Aug 16 '21 at 12:30
  • Does this answer your question? [Why does headless need to be false for Puppeteer to work?](https://stackoverflow.com/questions/63818869/why-does-headless-need-to-be-false-for-puppeteer-to-work) – ggorlen Apr 02 '22 at 00:04

1 Answers1

-1

Headless false:

This will be normal Auto UI test, which it launch the chrome browser, and you can see that the chrome browser window is open

Headless true:

Think of everything is running as UI TEST. However, this time it does not open the chrome browser physically, it only runs in command line, so no chrome window is open this time. This is very handy where you do not want to see in automatic running environment.

Xin
  • 33,823
  • 14
  • 84
  • 85
  • Why would any of this account for the different behavior between the two modes? I think OP already knows all of this. – ggorlen Aug 15 '21 at 13:58
  • 1
    Thanks for the answer, but as @ggorlen said, I'm aware of the different behavior between the two modes. My question is what is different on the technical level, like what args are different and how the website recognize me as a headless browser. – nadavshkoori Aug 15 '21 at 21:24