Python Selenium: element is not attached to the page document

Question

I am new in python and I am trying of obtain data about sold houses webpage. I need to click in each item (ad house), extract data of this, return initial webpage and click in another item....

In this moment it only print information of the first item and after get this error:

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: chrome=77.0.3865.90)

I tried with try and except, but it only prints the first item.

element = driver.find_elements_by_class_name('title-grid')
for j in range(0,len(element)):
    if element[j].is_displayed():
        element[j].click()
        product_containers = driver.find_elements_by_class_name('row.detailContent')
        print(product_containers)
        for container in product_containers:
            price=container.find_element_by_class_name('price').text
            #print(price)
            prices.append(price)
    j=j+1
    driver.back()
    time.sleep(15)

score 0 · Answer 1 · answered Oct 10 '19 at 21:20

Stale element stands for the case when the element you found is not member of the DOM anymore because the page has changed. You may find an item with the same attributes but it is not the same element you found. When you trigger element[j].click() I assume that a page load happens.

All your webelements in element = driver.find_elements_by_class_name('title-grid') becomes "stale" after the page load. But if that would not be the case, driver.back() will surely trigger a page load and renders all previously stored elements stale.

What you could do is:

either open the links in a new tab in your browser (and close them when you got the price)
or get the href or onclick attribute of your element list before navigating anywhere (when they are not stale yet), so you can navigate directly to the pages via driver.get(target)
if the page with the price is very simple, you may open the target url without selenium (ex. via requests) and parse the price from the received html response. No page refresh, no stale element problem.

Python Selenium: element is not attached to the page document

1 Answers1