18

I'm testing a site with lots of proxies, and the problem is some of those proxies are awfully slow. Therefore my code is stuck at loading pages every now and then. 

from selenium import webdriver

browser = webdriver.Firefox()
browser.get("http://example.com/example-page.php")
element = browser.find_element_by_id("someElement")

I've tried lots of stuff like explicit waits or implicit waits and been searching around for quite a while but still not yet found a solution or workaround. Nothing seems to really affect page loading line browser.get("http://example.com/example-page.php"), and that's why it's always stuck there.

Anybody got a solution for this?

Update 1:

JimEvans' answer solved my previous problem, and here you can find python patch for this new feature.

New problem: 

browser = webdriver.Firefox()
browser.set_page_load_timeout(30)

browser.get("http://example.com/example-page.php")

element = browser.find_element_by_id("elementA")
element.click() ## assume it's a link to a new page http://example.com/another-example.php

another_element = browser.find_element_by_id("another_element")

As you can see browser.set_page_load_timeout(30) only affects browser.get("http://example.com/example-page.php") which means if this page loads for over 30 seconds it will throw out a timeout exception, but the problem is that it has no power over page loading such as element.click(), although it does not block till the new page entirely loads up, another_element = browser.find_element_by_id("another_element") is the new pain in the ass, because either explicit waits or implicit waits would wait for the whole page to load up before it starts to look for that element. In some extreme cases this would take even HOURS. What can I do about it?

Paul T.
  • 714
  • 1
  • 9
  • 20
Shane
  • 4,875
  • 12
  • 49
  • 87

2 Answers2

12

You could try using the page load timeout introduced in the library. The implementation of it is not universal, but it's exposed for certain by the .NET and Java bindings, and has been implemented in and the Firefox driver now, and in the IE driver in the forthcoming 2.22. In Java, to set the page load timeout to 15 seconds, the code to set it would look like this:

driver.manage().timeouts().pageLoadTimeout(15, TimeUnit.SECONDS);

If it's not exposed in the Python language bindings, I'm sure the maintainer would eagerly accept a patch that implemented it.

JimEvans
  • 27,201
  • 7
  • 83
  • 108
  • Cool! Certainly a nice feature to have. – Ashwin Prabhu May 25 '12 at 12:00
  • @JimEvans: Thanks mate, I've just found python patch for this http://code.google.com/p/selenium/issues/detail?can=2&start=0&num=100&q=&colspec=ID%20Stars%20Type%20Status%20Priority%20Milestone%20Owner%20Summary&groupby=&sort=&id=687, and it works great. – Shane May 25 '12 at 12:24
  • @JimEvans: Hey mate, please check my update. There's a new problem bothering me. – Shane May 25 '12 at 14:27
  • @JimEvans: You introduced `pageLoadTimeout()` in IE? Why has no one put it into the changelog? Hell yeah, I was waiting for this for ages :) – Petr Janeček Jun 13 '12 at 22:09
  • There wasn't a changelog specific to the IE driver until 2.22, since there was no standalone mechanism for distribution of the IE bits. There is now, where every change is documented, but since 2.22.0 was the baseline, it didn't include every change from the entire history of the project as its first entry. – JimEvans Jun 14 '12 at 00:10
0

You can still speedup your script execution by waiting for presence (not waiting for visibility) of expected element for say 5-8 sec and then sending window.stop() JS Script (to stop loading further elements ) without waiting for entire page to load or catching the timeout exception for page load after 5-8 seconds then calling window.stop()

Because if the page not adopted lazy loading technique (loading only visible element and loading rest of element only after scroll) it loads each element before returning window.ready state so it will be slower if any of the element takes longer time to render.

Jlearner
  • 573
  • 4
  • 6