Selenium Python: How to wait for a page to load after a click?

Question

I want to grab the page source of the page after I make a click. And then go back using browser.back() function. But Selenium doesn't let the page fully load after the click and the content which is generated by JavaScript isn't being included in the page source of that page.

element[i].click()
#Need to wait here until the content is fully generated by JS.
#And then grab the page source.
scoreCardHTML  = browser.page_source
browser.back()

you could select a specific element and then wait for it to appear before you do your actions, thus letting the entire page load as well. However there may be a better way to this — VlassisFo, Jun 13 '16 at 10:40
Did you google your own question? I did and the first result is your exact question... http://www.obeythetestinggoat.com/how-to-get-selenium-to-wait-for-page-load-after-a-click.html — JeffC, Jul 05 '16 at 03:51
See also [Python Selenium - Wait until next page has loaded after form submit - Stack Overflow](https://stackoverflow.com/questions/42069503/python-selenium-wait-until-next-page-has-loaded-after-form-submit) which has some other (may be better) methods. — user202729, Dec 09 '21 at 08:39

score 15 · Answer 1 · edited Sep 11 '18 at 12:17

15

As Alan mentioned - you can wait for some element to be loaded. Below is an example code

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Firefox()
element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, "element_id")))

edited Sep 11 '18 at 12:17

Vivek

322
1
13

answered Jun 13 '16 at 10:47

wisnia

646
5
14

Can I wait for the presence of element by class name too? 'By.CLASS_NAME'... is that the right syntax to address class names? – abhanan93 Jun 13 '16 at 10:51
@abhanan93 You can wait by anything you can use to find an element. – RemcoW Jun 13 '16 at 11:37
if you expect more elements with that class you can use presence_of_all_elements_located instead – wisnia Jun 13 '16 at 11:41
I am facing this error message while using explicit wait even though the page is now fully loaded and I have fetched the HTML completely. TimeoutException: Message: Stacktrace: at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpQIQukJ/extensions/fxdriver@googlecode.com/components/driver-component.js:10770) – abhanan93 Jun 13 '16 at 15:57
Can you post your whole code here or pastebin if too long? – wisnia Jun 14 '16 at 06:14

score 4 · Answer 2 · answered Jun 22 '20 at 03:51

4

you can also use seleniums staleness_of

from selenium.webdriver.support.expected_conditions import staleness_of

def wait_for_page_load(browser, timeout=30):
    old_page = browser.find_element_by_tag_name('html')
    yield
    WebDriverWait(browser, timeout).until(
        staleness_of(old_page)
    )

answered Jun 22 '20 at 03:51

varinder singh

41
2

2

Whats with the yield statement? – wamster Oct 03 '20 at 04:58

score 3 · Answer 3 · answered Nov 04 '18 at 13:11

3

You can do it using this method of a loop of try and wait, an easy to implement method

from selenium import webdriver
browser = webdriver.Firefox()
browser.get("url")
Button=''
while not Button:
    try:
        Button=browser.find_element_by_name('NAME OF ELEMENT')
        Button.click()
    except:continue

answered Nov 04 '18 at 13:11

ahmed abdelmalek

356
3
8

3

Infinite loops are rarely the answer. – SMKS Jan 26 '21 at 16:36

score 0 · Answer 4 · answered Apr 17 '20 at 18:48

Assuming "pass" is an element in the current page and won't be present at the target page. I mostly use Id of the link I am going to click on. because it is rarely present at the target page.

while True:
  try:
    browser.find_element_by_id("pass")
  except:
    break

Selenium Python: How to wait for a page to load after a click?

4 Answers4

Linked