1

I am asking for generally checking if all elements of a page has been loaded. Is there a way to check that basically?

In the concrete example there is a page, I click on some button, and then I have to wait until I click on the 'next' button. However, this 'Next' button is available, selectable and clickable ALL THE TIME. So how to check with selenium that 'state'(?) of a page?

As a reminder: This is a question about selenium and not the quality of the webpage in question....

Alex
  • 41,580
  • 88
  • 260
  • 469
  • Can you give us the URL and your code up to the point where you need to wait? – Imran Dec 15 '17 at 12:51
  • Its a non-public page unfortunately. But I asked a general question, which should have a general answer ... – Alex Dec 15 '17 at 12:52
  • 1
    The general answer is that there is no general answer for your general problem. There can be specific answers that could build into a general answer for a specific website or set of websites. – BoboDarph Dec 15 '17 at 13:53
  • 1
    No Selenium doesn't provide a way to wait for all the elements for the simple reason that this state becomes rather subjective when the page is constructed dynamically via JavaScript (see lazy loading, ajax). What Selenium does by default is waiting for the static HTML to be completely loaded (`document.readyState === 'complete'`). To wait for some elements to be refreshed, use a waiter to wait for an element to be different from the one before the Click. – Florent B. Dec 15 '17 at 14:55

3 Answers3

3

As your question is about if there is a way with python-selenium to wait until all elements of a page has loaded, the Answer is No.

An Alternate Approach

Fundamentally, you can write a line of code (or implement it through a function) to check for the 'document.readyState' == "complete" as follows :

self.driver.execute_script("return document.readyState").equals("complete"))

But this approach have a drawback that it doesn't accounts for the JavaScript / AJAX Calls to be completed.

Why No

Writing the above mentioned line to wait for Page Loading is implemented by default through Selenium. So rewriting the same is a complete overhead. The client (i.e. the Web Browser) will never return the control back to the WebDriver instance until and unless 'document.readyState' is equal to "complete". Once this condition is fulfilled Selenium performs the next line of code.

It's worth to mention that though the client (i.e. the Web Browser) can return back the control to the WebDriver instance once 'document.readyState' equal to "complete" is achieved, it doesn't guarantees whether all the WebElements on the new HTML DOM are present, visible, interactable and clickable.

So, as per our requirement, if the next *WebElement with which you have to interact is not interactable by the time 'document.readyState' equal to "complete" is achieved, you have to induce WebDriverWait inconjunction with a matching expected_conditions with respect to the individual WebElement. Here are a few of the most used expected_condition:

  • element_to_be_clickable(locator)
  • presence_of_element_located(locator)
  • visibility_of(element)

References

You can find a couple of relevant discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Most of this is not correct. 1. readyState == complete doesn't mean all elements are loaded. It does not take into account any dynamically loaded elements which may still be loading. 2. Selenium does not always wait for readyState == complete. It only does so when it fetches a URL... not on clicks or any other action that triggers navigation or dynamic loading. – JeffC Dec 15 '17 at 15:50
  • 1
    @JeffC I think you got pretty much confused here. **Questions for you to Answer** : 1. Where did you find **`readyState == complete mean all elements are loaded`** within the context of my answer? 2. Show me any documentation to support your comment `Selenium does not always wait for readyState == complete` Definately the question was about featching URL. 3. Dynamic loading context is still not a part of the Question/Answer. – undetected Selenium Dec 15 '17 at 15:56
  • 1
    1. The question is how to... "wait until all elements of a page has loaded?" Your answer is Yes and you provided the code to check for ready state == complete. 2. Use Selenium. You don't need documentation to know that this is the case. Write a script that clicks a button and then clicks a button on the next page with no waits and no implicit waits. It will fail because the following element doesn't exist... that's because Selenium doesn't wait. If you want something specific, read this https://github.com/SeleniumHQ/selenium/issues/2626. – JeffC Dec 15 '17 at 19:57
  • No where in the question does it state anything about fetching a URL... what it does state is, "In the concrete example there is a page, I click on some button, and then I have to wait until I click on the 'next' button." which is exactly the example I just gave. If Selenium automatically waited in this case, why would the OP ask this question? Because it doesn't wait. It only waits on fetching a URL. 3. There's nothing in the question that states that dynamic loading is not a part of the question... why would you assume otherwise? It's extremely common these days on modern sites. – JeffC Dec 15 '17 at 19:59
0

Reliably determining whether a page has been fully loaded can be challenging. There is no way to know if all the elements have been loaded just like that. You must define some "anchor" points in each page so that as far as you aware, if these elements has been loaded, it is fair to assume the whole page has been loaded. Usually this involves a combination of tests. So for example, you can define that if the below combination of tests passes, the page is considered loaded:

  • JavaScript document.readyState === 'complete'
  • "Anchor" elements
  • All kinds of "spinners", if exist, disappeared.
Eugene S
  • 6,709
  • 8
  • 57
  • 91
  • So the webdriver cannot 'ask' the browser? The broswer must 'know'? Or is there some magic involved? – Alex Dec 15 '17 at 12:58
  • 1
    Well, I guess the way the browser "knows" is by expecting the `readyState` status. Once it becomes `complete`, the page has fully loaded. – Eugene S Dec 15 '17 at 13:13
  • 2
    Fully loaded does not mean that every element on the page finished doing it's business and can be interacted with. – BoboDarph Dec 15 '17 at 13:55
  • Complete doesn't apply to dynamic elements, AJAX calls, and the like. – JeffC Dec 15 '17 at 20:01
  • Agree with all the comments. This is why in my answer I said that this is a combinations of things where `readyState` is just one of them. – Eugene S Dec 16 '17 at 00:33
  • 1
    @EugeneS Your comment above, `Once it becomes complete, the page has fully loaded.` seemed like you didn't understand that complete doesn't always mean complete. – JeffC Dec 16 '17 at 06:27
0

There is something called the document.readyState which you can retrieve by executing a JavaScript script via Selenium. This doesn't work for dynamically loaded content. It returns one of these three states:

Loading The document is still being loaded, no css or other resources are available

Interactive The document has been loaded, no css or other resources are available

Complete Both the document and the css / other resources are loaded

You're looking for at least Interactive. You can retrieve the state by calling execute_script:

driver.execute_script("return document.readyState")
Marcel
  • 1,443
  • 2
  • 14
  • 24