0

I am trying to automate a website that can detect chromedriver. I have used a user agent, I have gone in and changed the cdc_ part of the chromedriver, and several other things. No matter what I do it detects chromedriver and won't do anything.

However, I have been able to use Subprocesses and open my normal browser and sendkeys to go to the website, and do some of the things I need, so I know it will work in my normal browser.

Is there any way I can launch regular chrome and attach selenium to it in order to automate tasks?

Lzypenguin
  • 945
  • 1
  • 7
  • 18

1 Answers1

0

Chromedriver drives chrome. It's not it's own browser.

Open chrome manually and go to chrome://version/

Then open your webdriver and go to the same link. You'll see that selenium is starting chrome on a new/temporary profile and enables a lot more arguments.

Just this line with no options driver = webdriver.Chrome() for me (in python) launches chrome with a long list of command line arguments: chrome selenium command line arguments

You're most likely being detected because of the way selenium works.

This is a great post for detailing the problem and potential things to try: Can a website detect when you are using selenium with chromedriver?

Sites can detect some of the variables set by selenium.

@Erti-Chris Eelmaa put it really well:

the way the selenium detection works, is that they test for pre-defined javascript variables which appear when running with selenium. The bot detection scripts usually look anything containing word "selenium" / "webdriver" in any of the variables (on window object), and also document variables called $cdc_ and $wdc_. Of course, all of this depends on which browser you are on. All the different browsers expose different things.

You can disable javascript, and disable the "--enable-automation" through experimental tags but this can impact the functionality of selenium and your script might not work.

It might not be what you want to hear but when you're working on this (with selenium) keep in mind that the site is designed to stop automation so you're facing an uphill battle with it.

RichEdwards
  • 3,423
  • 2
  • 6
  • 22
  • Thank you for this response. So when I do driver = webdriver.Chrome() it still opens the chromedriver app. When I move the chromedriver app out of my folder it errors. When I do driver = webdriver.Chrome('C:PATH\\TO\\Chrome.exe') It will open chrome, but then when I do driver.get(webpage) it never goes to the website. But with the browser window that is opened, if i manually go to the site it works just fine (I assume because its using CHROME and not chromedriver). – Lzypenguin Aug 18 '20 at 15:42
  • It seems like if I open CHROME and not Chromedriver with the driver = webdriver.Chrome() then the script just hangs up, and it never moves on. Is there a way to change my driver = webdriver.Chrome('C:\\Path\\To\\chrome.exe') line so that it opens chrome as my driver, but then continues with the script? – Lzypenguin Aug 18 '20 at 15:46
  • Open chromedriver through your script and have a look at task manager. You run chromedriver.exe and that starts chrome.exe. You'll see both processes are running. Chromedriver is a few mb, but chrome is massive in comparison. Chromedriver is your interface to controlling chrome. You could say Its driving chrome :-) – RichEdwards Aug 18 '20 at 16:09
  • Have a look at the Chromedriver website it says "WebDriver is an open source tool for automated testing of webapps across many browsers. It provides capabilities for navigating to web pages, user input, JavaScript execution, and more. " https://chromedriver.chromium.org/ – RichEdwards Aug 18 '20 at 16:10
  • I understand that. But when I have chromedriver open chrome, and I go to www.kroger.com and do anything vs when I open chrome directly (without chromedriver) I get completely different results. That website knows when you are using automation software. That is what I am trying to avoid. I wrote a script with VBA through excel a couple years ago that was able to bring to the front an already opened browser, and interact with the browser. I was able to use CSS selectors and click buttons and automate stuff. That is what I am trying to do with python..... There has to be a way. – Lzypenguin Aug 18 '20 at 16:13
  • Sure buddy. Where there's a will there's a way. Can you share what your overall goal is? I ask as it influences what I say next. A simple sign up script is one thing but a complex Multipage scraper or full test suite is another story – RichEdwards Aug 18 '20 at 16:58
  • I need to create an account with a specific website. Then I need to go my account page and scrape the new account number, as well as enter specific information in one of the fields on this page, and then check to make sure the information I entered was applicable (not already in use). – Lzypenguin Aug 18 '20 at 19:25