RSelenium StaleElementReference error in lapply

Question

I am trying to scrape a link with Rselenium. Sometimes--only sometimes and this is not replicable (cause when I re-run the code, the problem disappear)--the programme gives me an error as follows:

Error:   Summary: StaleElementReference
     Detail: An element command failed because the referenced element is no longer attached to the DOM.
     class: org.openqa.selenium.StaleElementReferenceException
     Further Details: run errorDetails method

I think this is because I clicked on a web element and the DOM is somehow modified after a click (see this answer: RSelenium throwing StaleElementReference error). In this case, my code is to click on all the "expand" arrow of a weblink so show the full text. But the click under concern here is wrapped in a sapply function as follows, so I cannot relocate the web element every time:

  arrow = remDr$findElements(using = 'class', value = "WB_text_opt")    #locate the arrows
  sapply(arrow, function(x){
    Sys.sleep(0.15)
    x$clickElement()
    })                  # click on them
  remDr$findElement('css', 'html')$sendKeysToElement(list(key = "end"))   # scroll the webpage down

score 2 · Answer 1 · answered Apr 22 '20 at 05:06

The selenium reference of the items in the arrow list will be refreshed when you click on the arrow list item in your function. That's the reason why your are getting the staleElementException. Please get the specific element in the loop using the xpath/get the elements and then point to specific arrow item using the index and then click on it.

sapply(arrow, function(x){
    Sys.sleep(0.15)
    x$clickElement() #<== This line will work only for the first iteration.
                     # you will get issue from the 2nd item as the element references
                     # updates, when you click on 1st item.
                     # Try using something like .findElements()[index]
    })

RSelenium StaleElementReference error in lapply

1 Answers1