4

I'm trying to scrape some stuff using Selenium/headless chrome. In some pages, the driver will not move to the next step until the full page is loaded (even though the relevant elements are present).

I tried setting the pageLoadStrategy capability like so:

caps = Selenium::WebDriver::Remote::Capabilities.chrome("chromeOptions" => {"args" => [ "--headless" ]})
      caps['pageLoadStrategy']='eager'
self.driver||=Selenium::WebDriver.for :chrome, :desired_capabilities =>  caps

I get the following error:

Selenium::WebDriver::Error::UnknownError: unknown error: cannot parse capability: pageLoadStrategy
from unknown error: page load strategy unsupported

Is there a way to make this work with chrome?

According to this post, pageLoadStrategy is supported, but I can't get the 'eager' option to work.

Page load strategy for Chrome driver

If not, are there more tried and true ways of doing this. I've been having a lot of trouble with Chrome (it also doesn't support unexpectedAlertBehaviour)

Gabriel
  • 157
  • 3
  • 11
  • 6
    The eager strategy is not yet supported ([source](https://cs.chromium.org/chromium/src/chrome/test/chromedriver/capabilities.cc?rcl=04235e62d2eae12e49b31ee74d8e6a448e3d8cc5&l=185)). Use `none` and wait for `document.readyState` to be `interactive`. – Florent B. Dec 18 '17 at 23:47
  • 1
    Thanks @FlorentB. Whats the best way to implement that wait (document.readyState interactive) . Also, looking at the source code I see that unexpectedAlerBehaviour is implemented. I tried this extensively like 3 weeks ago with no luck. Any thoughts on how to make that work? I should prob post a new question with that. – Gabriel Dec 19 '17 at 15:56

5 Answers5

3

Chrome does not support PageLoadStrategy.EAGER. But you can try PageLoadStrategy.NONE. Sometimes you need to synchronize driver wait for some elemnts but possibility is less.

Rohit Borse
  • 75
  • 1
  • 10
2

ChromeDriver 77.0 (which supports Chrome version 77) now supports eager as pageLoadStrategy.

Resolved issue 1902: Support eager page load strategy [Pri-2]

See release notes

rudevdr
  • 389
  • 1
  • 5
  • 15
  • So I have ChromeDriver 77 but still getting this message `invalid argument: 'pageLoadStrategy=eager' not yet supported` – Pramod Oct 25 '19 at 08:12
1

For those who are waiting for the "eager" feature. You can check the "CheckSupport" function in the source code.

It seems it is still not supported even for now.

Ru Lu
  • 11
  • 1
  • 1
1

https://cs.chromium.org/chromium/src/chrome/test/chromedriver/chrome/page_load_strategy.cc?q=page_load_strategy&sq=package:chromium&dr=C

The corresponding page load strategy for EAGER is not yet supported by chrome.

Bipin Kumar Chaurasia
  • 372
  • 1
  • 13
0

The eager strategy now seems to be supported by chrome since v66 (source).

Eimantas G
  • 1
  • 1
  • 2