7

I'm attempting to use a headless chrome browser with selenium that also bypasses the bot detection test and currently using the the following project https://github.com/ultrafunkamsterdam/undetected-chromedriver Every time I try to implement the code it doesn't recognise the driver. Here is the link for you to understand

Here is the code

#
# UNDETECTED chromedriver (headless,even)
#
import undetected_chromedriver as uc
options = uc.ChromeOptions()
options.headless=True
options.add_argument('--headless')
chrome = uc.Chrome(options=options)
chrome.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')
chrome.save_screenshot('datadome_undetected_webddriver.png')

So when I use chrome.get() I receive an error as chrome has no get() member. I've installed the project using the pip command as well. So I was thinking do I need to direct the path to the chromedriver and where would that be because I doubt it'll be the normal driver and the documentation never mentioned the PATH for the driver.

Okay so when I run the program I get the following in terminal

DevTools listening on ws://127.0.0.1:55903/devtools/browser/ef3a54cf-35b9-400f-972c-2b54ca227eb8
[0102/000855.199:INFO:CONSOLE(2)] "JQMIGRATE: Migrate is installed, version 1.4.1", source: https://datadome.co/wp-content/cache/busting/1/wp-includes/js/jquery/jquery-migrate.min-1.4.1.js (2)
[0102/000856.946:INFO:CONSOLE(1)] "Messaging child iframes", source: https://track.gaconnector.com/gaconnector.js (1)
[0102/000856.946:INFO:CONSOLE(1)] "https://track.gaconnector.com/track_pageview?gaconnector_id=ddade4fc-93d0-20a3-79fa-39648d8e6629&account_id=6dd3433635353fd00f486550bcd5b157&referer=&GA_Client_ID=183291439.1609510136&page_url=https%3A%2F%2Fdatadome.co%2Fcustomers-stories%2Ftoppreise-ends-web-scraping-and-content-theft-with-datadome%2F&gclid=&utm_campaign=&utm_term=&utm_content=&utm_source=&utm_medium=", source: https://track.gaconnector.com/gaconnector.js (1)
PS D:\Programming\Python> [0102/000902.158:INFO:CONSOLE(0)] "The resource https://js.driftt.com/core/assets/js/runtime~main.a73a2727.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.", source: https://js.driftt.com/core?embedId=2rce7xnshapc&forceShow=false&skipCampaigns=false&sessionId=98163ad1-ed91-459e-9473-3f8861aa717e&sessionStarted=1609510138&campaignRefreshToken=107a7fd5-edb4-499b-9f39-5306c189cdb6&pageLoadStartTime=1609510135613 (0)
[0102/000902.272:ERROR:web_contents_delegate.cc(224)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
[0102/000902.272:ERROR:web_contents_delegate.cc(224)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
[0102/000902.475:INFO:CONSOLE(0)] "The resource https://js.driftt.com/core/assets/js/runtime~main.a73a2727.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.", source: https://js.driftt.com/core/chat (0)
[0102/000906.041:INFO:CONSOLE(0)] "The resource https://js.zohocdn.com/ichat/js/73291e5e_wmsbridge.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.", source: https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/ (0)
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Code Collector
  • 97
  • 1
  • 1
  • 8
  • I've added chrome driver to the variable PATH and still nothing. I've even have it as parameters "options = uc.ChromeOptions('D:/Programming/chromedriver.exe')" – Code Collector Jan 01 '21 at 14:12
  • always put full error message (starting at word "Traceback") in question (not commnt) as ext not screenshot). There are other useful information. – furas Jan 01 '21 at 16:00

2 Answers2

14

ChromeOptions() is defined within selenium.webdriver.chrome.options but not within undetected_chromedriver.

Solution

You can use the following solution:

  • Code Block:

    import undetected_chromedriver as uc
from selenium import webdriver

options = webdriver.ChromeOptions() 
options.headless = True
driver = uc.Chrome(options=options)
driver.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')
driver.save_screenshot('datadome_undetected_webddriver.png')
driver.quit()
print("Program Ended")

  • Console Output:

undetected_chromedriver

  • Screenshot:

datadome_undetected_webddriver

References

You canfind a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • 3
    For future readers: in the newest version 2.2.1, I can just use ChromeOptions from UC without any problem, so we don't need to use chrome option from Selenium. There are fixes in ChromeOptions of UC, so we should use this option instead of Selenium. – jackblk Apr 07 '21 at 10:29
1

ChromeOptions is defined in the undetected_chromedriver now. You can access it as:

undetected_chromedriver.options.ChromeOptions()
William
  • 464
  • 2
  • 16