5

I have a problem, that https://bet365.com started to block Selenium bots. It's stuck on the white screen.

I've tried everything, now my code looks like:

username = os.getenv("USERNAME")
userProfile = "C:\\Users\\" + username + "\\AppData\\Local\\Google\\Chrome\\User Data\\Default"

opts = webdriver.ChromeOptions()

opts.add_argument("user-data-dir={}".format(userProfile))
opts.add_argument('--profile-directory=Profile 2')

opts.add_argument("user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36'")

opts.add_experimental_option('useAutomationExtension', False)
opts.add_experimental_option("excludeSwitches", ["enable-automation", "ignore-certificate-errors", "safebrowsing-disable-download-protection", "safebrowsing-disable-auto-update", "disable-client-side-phishing-detection"])

opts.add_argument("--disable-notifications")
opts.add_argument("--disable-blink-features")
opts.add_argument("--disable-blink-features=AutomationControlled")
opts.add_experimental_option("excludeSwitches", ["enable-automation"])
opts.add_experimental_option('useAutomationExtension', False)

self.browser = webdriver.Chrome(options=opts, executable_path=self.resource_path("chromedriver.exe"))

window_size = self.browser.execute_script("""
        return [window.outerWidth - window.innerWidth + arguments[0],
          window.outerHeight - window.innerHeight + arguments[1]];
        """, width, height)
self.browser.set_window_size(*window_size)

self.browser.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
    "source": """
        Object.defineProperty(navigator, 'webdriver', {
        get: () => undefined
    })
    """
})

self.browser.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
    "source": """
        Object.defineProperty(navigator, 'languages', {
        get: () => ["en-US", "en", "es"]
    })
    """
})

self.browser.execute_cdp_cmd("Network.enable", {})
self.browser.execute_cdp_cmd("Network.setExtraHTTPHeaders", {"headers": {"User-Agent": "browser1"}})

self.browser.execute_cdp_cmd('Network.setUserAgentOverride',
            {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36'})

Also I tried to launch bot on Firefox using geckodriver, and tried to launch it using phantomjs, but it also have no results. And the last, I tried to obfuscate JavaScript code in chromedriver.exe by changing $cdc_ to smth else ($aba_), bot launhes, all is ok, but the same no access to Bet365.

After launching programm browser is stuck on this: Bet365 Screenshot

And when I go to the website manually by myself, all is ok, it loads.

Can anyone help me with this problem?

Skat
  • 51
  • 2
  • It is easy for orginal site to detect automation tool.But I share yor entusiasam, bet 365 blocked my 5 accounts :-).There are a lot of links on this site how to avoid that site detect that you are using automation tool. – Gaj Julije Nov 28 '20 at 20:26
  • 1
    @GajJulije But I tried all ways to avoid detection using links on this site) And all these ways are now in code on top. Did I correctly understood you, that there are some ways to avoid detection, that are not listed in my code? If so, can you share them with me, please?) – Skat Nov 28 '20 at 21:18
  • Same problem here. It was working fine until this morning, but the bet365 changed something in server-side to block automations, i believe. I have alread tried with chromedriver, gecko, different selenium versions, but nothing seems to work. That's a very recent issue. – rodrigorf Nov 28 '20 at 22:07
  • 1
    The issue mentioned in this link (Chrome driver for Selenium stuck in grey screen on bet365 site), it`s not the same issue, different errors but looks like the same. – rodrigorf Nov 28 '20 at 22:20
  • @rodrigorf, Looking for reply from you, if you will find a solution) – Skat Nov 29 '20 at 06:21
  • 1
    Did anyone find a solution? – user2412672 Nov 29 '20 at 12:00
  • No solutions so far, i have already run a couple of tests, including Chrome Canary v89, no changes. First load always freezes in infinite loop. If i block javascript the background load correctly but it turns unable to load the rest of components. The selenium GET function its providing some header information that allows bet365 to detect automation. I don't know yet what is it. – rodrigorf Nov 29 '20 at 19:54
  • Found the solution in this link: [link](https://stackoverflow.com/questions/65071572/out-of-memory-error-in-pythonselenium-first-load). It worked for me. – rodrigorf Dec 11 '20 at 17:55
  • @rodrigorf, Is it working now? I've found the same solution, that you suggested, but now everything is the same by some reason, the site is not loading again. – Skat Jan 29 '21 at 18:46
  • Hey @Danny, yeah the cdc solution works, but stay alert for different "gray screens" problem, as far as i know there is at leat two different blocking screens with different causes, be sure that the IP or IP range of the server you are running the robot it`s not blocked. You can check accessing manually the bet365 website. And change de .exe file using notepad++, i have tried with vsCode but it didnt worked, something related with encoding. – rodrigorf Jan 31 '21 at 17:05
  • @rodrigorf Yes, I checked everything, manually the site is working, but program with modified chromedriver can't get access to the site. cdc_ is changed, and before last block everything worked with this chromedriver. Can you now get access to bet365 with some code or no? – Skat Feb 02 '21 at 16:25
  • Hi @Danny, yes the code i have tested its working just fine. :) – rodrigorf Feb 09 '21 at 23:42
  • Hi @rodrigorf, would you be kind to share example of code, launching the chrome? (or chromedriver.exe, if the issue is in it additionally to changing all cdc_ variables) – Skat Feb 11 '21 at 15:33
  • Hi @Danny, sure, i`ll see what i can do and send you an example. talk to you soon – rodrigorf Feb 11 '21 at 23:31

0 Answers0