1

My Problem:

I am trying to scrape a webpage

#start selenium
selServ <- selenium()
# check ports
selServ$log()$stderr 
# conect using port
sel <- remoteDr(browserName = "chrome", port = 4567) 
# go to the URL 
sel %>% 
go("https://jurispub.admin.ch/publiws/pub/search.jsf")
# hit search button
sel %>% 
findElement("name", "form:searchSubmitButton") %>%  # find the submit button 
  elementClick() # click it

Next, I use a loop to get some data, at the end of the loop I let Selenium click on the next button:

i <- 1
while (i <= nr+1) {
Sys.sleep(0.5)
sel %>% 
getPageSource() %>% # like read_html()
html_node("table.iceDatTbl") -> dtbl  # this is the data table
sel %>%
findElement("xpath", ".//img[contains(@src, 'arrow-next')]/../../a") %>% 
elementClick() # go to next page}

However, after a certain time (random amount of loops) I get this error message:

Error detected:
Response status code : 200 
Selenium Status code:  10 

Selenium Status summary:  StaleElementReference 
Selenium Status detail:  An element command failed because the referenced element is no longer attached to the DOM. 
Selenium message:  stale element reference: element is not attached to the page document
  (Session info: chrome=69.0.3497.100)
  (Driver info: chromedriver=70.0.3538.16 (16ed95b41bb05e565b11fb66ac33c660b721f778),platform=Windows NT 10.0.17134 x86_64) 
Please check the response with errorResponse()
Please check the content returned with errorContent()

> errorContent()

[1] "stale element reference: element is not attached to the page document\n

My approach

I think clicking the next button is causing the error, since this is the only click in the loop

sel %>%
        findElement("xpath", ".//img[contains(@src, 'arrow-next')]/../../a") %>% 
        elementClick() # go to next page

I found this post and I think i have a similar problem: RSelenium throwing StaleElementReference error

However, I do not know how to implement it in my code.

Update

This is slightly different from from StaleElementReference Exception in PageFactory since I've already had a findElement() implemented before elementClick() My question is more similar to RSelenium cannot access DOM

and I tried to implement this solution, i.e a JavaScript click:

 # go to next page  
  sel %>% findElement("xpath", ".//img[contains(@src, 'arrow-next')]") -> resu
  executeScript(sel,"arguments[0].click();", args=list(resu))

However, I still get "random" StaleElementReference errors.. (less frequently) :,( Could there anything else in my code cause this error?

captcoma
  • 1,768
  • 13
  • 29
  • 1
    I updated the question. The solution from the post which is marked as duplicate was already implemented in the initial question. – captcoma Oct 21 '18 at 00:11

0 Answers0