Selenium error "Element is no longer attached to the DOM" while scraping data

Question

        for i in driver.find_elements_by_class_name("endorse-count"):
            try:
                i.click()
            except:
                continue

            elem = WebDriverWait(driver, 100).until(EC.presence_of_element_located((By.CLASS_NAME, "dialog-window")))
            src = elem.get_attribute("innerHTML")

            add_skill(name, src)

            WebDriverWait(driver, timeout=10)

I'm getting the following error while running the above code -

selenium.common.exceptions.StaleElementReferenceException: Message: u'Element is no longer attached to the DOM' ; Stacktrace: 
    at fxdriver.cache.getElementAt (resource://fxdriver/modules/web_element_cache.js:7646)

for line -

src = elem.get_attribute("innerHTML")

I'm running this code on LinkedIn user profile page, after logging in.

I tried putting the following line of code after "i.click()" -

driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

But then I see that function "add_skill(name, src)" is not called and none of the code after driver.manage() is called, though for loop and further i.click() work fine.

score 1 · Answer 1 · answered Sep 24 '13 at 08:19

Selenium is trying to complete actions (such as clicking a button or link) before verifying that the target element has rendered on the page. Selenium can be more patient, but you have to explicitly ask him to be.

For example, if you are testing something that makes an AJAX request, you can try something like this (in Ruby):

# timeout is in seconds
def wait_for_ajax(timeout=x)
 time_limit, interval = (Time.now + timeout), 0.5
 loop do
   break if @driver.execute_script "return jQuery.active == 0"
   sleep interval
   raise "Wait for AJAX timed out after waiting for #{timeout} seconds" if Time.now > time_limit
 end
end

To ensure your tests are fully comprehensive, always make Selenium waits for elements to load before running a task.

If you see carefully, I've already used wait in my code - `elem = WebDriverWait(driver, 100).until(EC.presence_of_element_located((By.CLASS_NAME, "dialog-window")))` but it isn't helping. — theharshest, Oct 14 '13 at 04:18

score 1 · Answer 2 · edited May 23 '17 at 10:30

1

I had faced a similar issue and tried refreshing the page before finding that element, and it worked...

driver.navigate().refresh();

Though I couldnt reason out how this worked.

If this works for you as well, please let me know. I just want to learn more about this exception.

you can refer this page to learn about a similar issue

edited May 23 '17 at 10:30

Community

1
1

answered Oct 14 '13 at 06:38

Amith

6,818
6
34
45

Thanks for the help, but the popup in which I'm working disappears when page is refreshed, so this won't help me. The popup here is generated by clicking on the number of endorsers of a particular user for a particular skill. I want to work on that popup, hence - `src = elem.get_attribute("innerHTML")` - I'm taking HTML for it. – theharshest Oct 15 '13 at 01:54

score 0 · Answer 3 · answered Feb 09 '15 at 21:46

I had a similar problem when trying to execute some javascript (IJavaScripExecutor). I created an IWebElement and passed that to the JSE and that failed for me. When I moved the driver.FindElement(BySelector) into my JSE call, then it worked. (C# code ahead.)

Instead of:

IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
IWebElement tableEl = driver.FindElement(selector);
js.ExecuteScript(script, tableEl);

I had to do:

IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
js.ExecuteScript(script, driver.FindElement(selector));

You may have to do something similar: move your selector or element creation onto the same line as what you are trying to do. Or, maybe, in your case:

src = driver.find_element_by_class_name("dialog-window").get_attribute("innerHTML")

Upon closer inspection, that's what looks to be your problem, there's a stale web element object when you try to use the get_attribute method.

Selenium error "Element is no longer attached to the DOM" while scraping data

3 Answers3