5

So, this is the code I'm having troubles with:

def scrap():
        options = webdriver.ChromeOptions();
        options.add_argument('headless');
        options.add_argument('--profile-directory=Profile 1')
        options.add_argument("--user-data-dir=C:/Users/omarl/AppData/Local/Google/Chrome/User Data/")
        options.add_argument("--remote-debugging-port=45447")
    
        options.add_argument("--disable-gpu") 
        browser = webdriver.Chrome(executable_path=r"C:\Users\omarl\OneDrive\Escritorio\chromedriver.exe", options=options)
        
        scrapURL = "https://es.wallapop.com/search?distance=30000&keywords=leggins&latitude=41.38804&longitude=2.17001&filters_source=quick_filters"
        browser.get(scrapURL)
        #...

And the error:

WebDriverException: unknown error: unable to discover open pages

I don't have any instances of chrome when I execute the script, and when I'm using it without the headless option it works fine. Any idea why this is happening? Please, note that I'm using the --remote-debuggin-port provided in similar questions.

I'm using ChromeDriver 86.0.4240.22

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Norhther
  • 545
  • 3
  • 15
  • 35
  • with the headless version also you are using the same executable path ? – Sariq Shaikh Nov 16 '20 at 07:54
  • Maybe a issue with your path. Try `options.add_argument(r'--user-data-dir="C:\Users\omarl\AppData\Local\Chromium\User Data\Default"')`. – Florent B. Nov 18 '20 at 12:43
  • @Norhther it's hard to give you an explicit answer, because the section of code that is potentially causing you the error is omitted from your question. Please provide additional details. – Life is complex Nov 20 '20 at 18:02

2 Answers2

2

To invoke a Chrome Profile in Headless mode you can use the --user-data-dir argument only and you can safely remove --profile-directory argument as follows:

  • Code Block:

    from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
# options.add_argument('--profile-directory=Profile 1')
options.add_argument(r"--user-data-dir=C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\Default")
options.add_argument("--remote-debugging-port=9222")
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.google.com/')
print("Chrome Headless launched")

  • Console Output:

    DevTools listening on ws://127.0.0.1:9222/devtools/browser/93c67c41-e125-4d12-abc0-fcf0f07a62f4
Chrome Headless launched

References

You can find a couple of relevant detailed discussions in:

Additional Considerations

Ensure that:

  • Selenium is upgraded to current released Version 3.141.0.
  • ChromeDriver is updated to current ChromeDriver v86.0 level.
  • Chrome is updated to current Chrome Version 86.0 level. (as per ChromeDriver v86.0 release notes).
  • Execute your @Test as non-root user.
  • Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

tl; dr

ChromeDriver remote debug port reservation race conditions

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • 2
    Here you're saying `you can use...`. So the problem with my code is that I was using the args `--profile-directory` and `--user-data-dir`? Answers should be explicit about what the error is. – Norhther Nov 16 '20 at 19:39
0

Have you tried using arg --no-sandbox? A lot of people on Chrome Driver Error using Selenium: Unable to Discover Open Pages have had success with that argument.

user7788539
  • 89
  • 1
  • 7
  • This was a good idea, but something else after "browser.get" is likely causing his issue. I'm upvoted your answer, because it was a solid attempt to help him solve his issue. – Life is complex Nov 22 '20 at 22:33