How to mimic like a human while using selenium webdriver

Question

I have used useragents, cookies, undetected_chromedriver and other things as well but still the website detect as a bot and redirecting me to the different page instead of original flow. When I normally login then its working fine

I have used this:

chrome_options = uc.ChromeOptions()
    chrome_options.add_argument(
        "user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.9999.99 Safari/537.36")
    # Adding argument to disable the AutomationControlled flag
    chrome_options.add_argument("--disable-blink-features=AutomationControlled")

    # Exclude the collection of enable-automation switches
    chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])

    # Turn-off userAutomationExtension
    chrome_options.add_experimental_option("useAutomationExtension", False)
    chrome_options.add_argument("--disable-notifications")
    chrome_options.add_argument("--disable-popup-blocking")
    chrome_options.add_argument("--enable-javascript")
    prefs = {"profile.default_content_setting_values.notifications": 2}
    chrome_options.add_experimental_option("prefs", prefs)
    driver = webdriver.Chrome(chrome_options=chrome_options)

I'm doing any data scraping but for automation. Please suggest some robust techniques to handle this situation.

It's a very broad question you're asking, you pretty much need to reverse engineer that website/service bot detection. Try different things: moving the mouse in a (recorded) human way, typing in a (recorded) human way, etc... — Learning is a mess, Jul 04 '23 at 09:51
Good luck, depending on how protective (= sensitive) the website is from bots, this could be a very hard task. Not trying to discourage anyone here. — Learning is a mess, Jul 04 '23 at 10:15

score 0 · Answer 1 · answered Jul 04 '23 at 12:20

this is the setting what I used, should try it

#https://stackoverflow.com/questions/53039551/selenium-webdriver-modifying-navigator-webdriver-flag-to-prevent-selenium-detec
edge_options.add_experimental_option('excludeSwitches', ['enable-automation'])
edge_options.add_experimental_option('useAutomationExtension', False)
edge_options.add_argument('lang=zh-CN,zh,zh-TW,en-US,en')
edge_options.add_argument('--disable-gpu')
edge_options.add_argument("disable-blink-features=AutomationControlled")
driver = webdriver.Edge(options=edge_options, service = service)
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})

How to mimic like a human while using selenium webdriver

1 Answers1