11

I'm having some trouble trying to access a web site (bet365.com) with a chrome driver and selenium (I'm quite being "blocked").

I can access the site with my ordinary chrome but when I try with chrome driver, it doesn't work.

I had this problem before and corrected it by using some options as below (python):

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'PATH_TO\chromedriver.exe')
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
  "source": """
    Object.defineProperty(navigator, 'webdriver', {
      get: () => undefined
    })
  """
})
driver.execute_cdp_cmd("Network.enable", {})
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})

driver.get("https://www.bet365.com/")

Now, the problem came back and this code is not working anymore to bypass the protection. Can someone help me?

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Ricardo Guerra
  • 111
  • 1
  • 1
  • 4
  • Maybe you're not supposed to do that? – Kelly Bundy Feb 07 '21 at 20:20
  • Does this answer your question? [last problem when scraping bet365.com with selenium](https://stackoverflow.com/questions/66089231/last-problem-when-scraping-bet365-com-with-selenium) – PDHide Feb 07 '21 at 22:13
  • Hello @Ricardo Guerra and undetected Selenium, your code actually worked for both bet365 and the site I was having trouble with. Thank you so much guys! – DerickMasai Jul 30 '23 at 13:26

1 Answers1

10

In case the Selenium driven ChromeDriver initiated Browsing Context is getting detected a potential solution would be to use the undetected-chromedriver to initialize the Chrome Browsing Context.

undetected-chromedriver is an optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io. It automatically downloads the driver binary and patches it.

  • Code Block:

    import undetected_chromedriver as uc
from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
driver = uc.Chrome(options=options)
driver.get('https://bet365.com')

References

You can find a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • It works, in the aspect that it avoids the Out of Memory error, but it is not possible to interact with the page.I mean, if you programmatically click on "In-Play", the page doesn't load. If you just open Bet365 with the In-Play link, it also doesn't work. – puppet Feb 08 '21 at 10:47
  • @puppet, did you find any solution? – Skat Feb 13 '21 at 11:44
  • 1
    No, nothing so far – puppet Feb 14 '21 at 19:55
  • Could this be an issue where a website is protected by recaptcha – Taio Nov 14 '22 at 15:22