Looping through pages and getting StaleElementReferenceException in python Selenium

Question

So I am looping through a bunch of webpages. And currently the webpages all have the same structure with a back button and a forwards button (//span/a)[2]. For some reason I can loop through the first page (and sometimes the second page). However I continue to get a StaleElementReferenceException.

Here is the related code:

for x in range(0,5):
    print 'page %d' %(x)
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.XPATH, "(//span/a)[2]"))
    )
    listItems = driver.find_elements_by_xpath("//td[@class='CourseCode']/a")
    for element in listItems:
        elementText = element.text
        print(elementText)
        writeFile.write(element.text + '\n')
    driver.find_element_by_xpath("(//span/a)[2]").click()

In particular here is the stack trace:

Traceback (most recent call last):
File "getList.py", line 21, in lookup
addListItems(driver, courseCodeFile)
File "getList.py", line 44, in addListItems
elementText = element.text
File "/home/francisco/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 73, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "/home/francisco/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
return self._parent.execute(command, params)
File "/home/francisco/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/francisco/.local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: Message: The element reference is stale. Either the element is no longer attached to the DOM or the page has been refreshed.

I've tried a bunch to no avail. The weird thing is, if not looping through I was able to get the function to work properly for two pages.

Of note prior to the RTE, it would print the text of the first 2-3 elements of listItems that were obtained in the previous page.

Naveen Kumar R B · Accepted Answer · 2017-02-28T13:51:20.133

1

You can avoid StaleElementReferenceExpection by using stalenessOf Expected Condition in WebDriverWait.

StaleElementReferenceExpection occurs in two common cases:

The element has been deleted entirely.
The element is no longer attached to the DOM.

As you are using common locators across all the web pages, once you click on an element, selenium is still referencing to the locators in the previous page (DOM is NOT yet updated, still referencing to the old web page)

One simple solution is to add time.sleep at the end of the code, so that DOM will get updated, and the locators will be applied on DOM of the new web page.

for x in range(0,5):
    print 'page %d' %(x)
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.XPATH, "(//span/a)[2]"))
    )
    listItems = driver.find_elements_by_xpath("//td[@class='CourseCode']/a")
    for element in listItems:
        elementText = element.text
        print(elementText)
        writeFile.write(element.text + '\n')
    driver.find_element_by_xpath("(//span/a)[2]").click()
    import time
    time.sleep(0.5) //0.5 seconds

Another solution is to check for unique elements in each web page, which may not be possible in a for loop (possible if you use if-else & indexing etc.)

edited Feb 28 '17 at 13:51

answered Feb 27 '17 at 07:11

Naveen Kumar R B

6,248
5
32
65

Just for some minor clarification. What is the point of the WebDriverWait then? As I understand this is to wait until that element is loaded. However in this case is it just still referencing the old DOM? – Francisco Feb 27 '17 at 14:29
1

Yes, WebDriverWait is to wait until some condition is met, like Presence_Of_Element, Visibiliti_Of_Element etc. In your case, as locators are same, which are present in both the DOMs of both current and previous pages, giving exception. look for more details here http://stackoverflow.com/questions/16166261/selenium-webdriver-how-to-resolve-stale-element-reference-exception & https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/support/ui/ExpectedConditions.html#stalenessOf-org.openqa.selenium.WebElement-. try with `stalenessOf` condition instead of sleep. – Naveen Kumar R B Feb 27 '17 at 15:00
Used `staleness_of` and it worked perfectly. If you want to update your answer to mention to use the `staleness_of` on the `(//span/a)[2]` after the `click()`. I'd be happy to choose this as a solution. – Francisco Feb 28 '17 at 13:48

Looping through pages and getting StaleElementReferenceException in python Selenium

1 Answers1