3

I've tried to run my python selenium bot in --headless mode. Unfortunately with --headless mode it is not bypassing Cloudflare bot protection. Any ideas what would work there?

options = uc.ChromeOptions()
options.headless=True
options.add_argument("--window-size=1920,1080")
options.add_argument('--ignore-certificate-errors')
options.add_argument('--allow-running-insecure-content')
options.add_argument('--headless')

scrap = uc.Chrome(use_subprocess=True,options=options)

screen of Cloudflare bot protection

Osc4r
  • 126
  • 1
  • 7
  • The Headless tag is just to do with gui rendering. You're still using chrome but without seeing it. You should consider the that if a website has anti-automation protection it doesn't want you using automation against it. A lot of clever developers spend a lot of time and money to specifically to stop you. Your best way around it is to remove the browser entirely, use something like python requests and craft you own user string. – RichEdwards Jul 09 '22 at 07:58
  • But with rendering, it does work and passes successfully. And I've read the TOS of the site and there is nothing about using (or not using) automated bots. Cloudflare probably is there to avoid DDOS attacks – Osc4r Jul 09 '22 at 08:07
  • 1
    Whatever... Just found that headless works only for naive bots. https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/258 – Osc4r Jul 09 '22 at 08:35

1 Answers1

1

Use a virtual display such as Xvfb so that you don't need to use headless mode on a headless machine such as Linux servers.

There's a Selenium Python framework, https://github.com/seleniumbase/SeleniumBase, with built-in integration to undetected-chromedriver for that exact thing. When running your tests, add --uc as a pytest command-line option for your SeleniumBase tests. Eg:

pytest --uc --xvfb

That lets you successfully run Selenium Python tests on a Linux headless machine in undetected-chromedriver mode. (With SeleniumBase)

Michael Mintz
  • 9,007
  • 6
  • 31
  • 48