2

Here is what I tried :

from selenium import webdriver
driver = webdriver.Chrome(executable_path='chromedriver.exe')
driver.get("https://secure07c.chase.com/web/auth/#/logon/logon/chaseOnline?")
username = driver.find_element_by_id("userId-text-input-field")

The problem I ran into is that when I simply execute this and then manually fill the fields and click login, an error page that is made for protecting from bots pops up.

When I remove the line username = driver.find_element_by_id("userId-text-input-field") the website works correctly and I can login manually from the automated selenium webdriver driven page. Same problem happens when doing driver.page_source and many other tests that request elements from the webpage.

I tried a lot of things (most options, flags, user agent, ...) but they are not relevant in this issue that's why I included the simplified version of the code that is causing the issue, basically any element selection.

The way selenium requests elements is suspicious I guess, simply finding elements raised suspicion in the chase bank website. I want to understand how selenium is finding / selecting elements and how anti-bots are detecting this very simple action. Is there a way around it ?

Elyes Lounissi
  • 405
  • 3
  • 12
  • 1
    When I attempted to access the login page via Selenium, I received this warning "We can't find that username and password. You can reset your password or try again." That was using pure automation. Is this the error you received? – Life is complex Apr 23 '21 at 12:19
  • @Lifeiscomplex nope, I get an overlay page destined to protection from bots, can you please share what you tried ? was it chrome 90 with chromedriver of that specific version ? and what was the code you tried ? is it the exact 4 lines that I shared then manually inputting the credentials or something else ? – Elyes Lounissi Apr 24 '21 at 00:27
  • 1
    I used chromedriver 90.x. I have over 50 lines of code. I don't have a Chase account to fully test the code. – Life is complex Apr 24 '21 at 00:40
  • Can you please share your code ? as my simple code is causing problems right after any `find_element` no matter what i add to the code, even if there are no actions after the `find_element` and I need it for testing some websites that implement similar security (distil network security), distil is fighting selenium so hard, you can look it up, I need to go past it and go undetected, this is my goal, not a particular website, but understanding and bypassing the security, Thank you so much in advance – Elyes Lounissi Apr 24 '21 at 01:09
  • I did some more testing and the code is having issues bypassing the bot protection, which I don't think is Distil Networks. I didn't see references to distil in the javascript files. If I figure something out I will let you know here. – Life is complex Apr 24 '21 at 17:37

1 Answers1

0

Chase can see that it is an automated script sending the request, not a real human sending it. Stack Overflow also uses similar technology. There is no way to escape this, otherwise, bots would be all over banking websites and DDoSing them.

Sergei Kiselev
  • 42
  • 6
  • Getting an element from a loaded webpage is probably done locally from the source code of the page, selenium parses the code and gives me back that element, how are they detecting that, what is selenium doing wrong ? – Elyes Lounissi Apr 20 '21 at 05:54
  • @ElyesLounissi The script still has to submit a query to Chase for the source code, which Chase's system detects not to be from a real webbrowser. – Sergei Kiselev Apr 20 '21 at 05:58
  • 1
    I believe @SergeiKiselev is correct, you need to hide the fact that you're using chrome webdriver instead of a standard chrome browser. This may be quite difficult on a banking site, but can be done if you're stubborn enough. this site is a tool for testing how your bot looks to a web server and could give you an idea of what they're flagging: https://bot.sannysoft.com/ this may give you some more ideas on how to avoid detection: https://medium.com/analytics-vidhya/the-art-of-not-getting-blocked-how-i-used-selenium-python-to-scrape-facebook-and-tiktok-fd6b31dbe85f – samthaman Apr 20 '21 at 06:12
  • 1
    @samthaman it is not the fact that chrome is using selenium webdriver, because if I do not perform any finding elements, and manually do the login it works, if I do a single `find_element` without doing anything to the element, just that single statement it stops working, how is finding elements affecting the page ? – Elyes Lounissi Apr 20 '21 at 17:57