1

I've been trying to crawl data from the website AlgoExplorer. It has a table with pagnigation to store data. Even though I use an Explicit Wait for clicking a 'next' button, it still get StaleException. Here is a piece of my code, and an image of error:

for i in tqdm(range(5)): 
  page = driver.find_element_by_tag_name('tbody').find_elements_by_tag_name('a')
  for e in page:
    pages.append(e.text)
  WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.pagination.next'))).click()

Of course, every variable has been declared and library has been imported.

Can you please explain me why I still have that exception? enter image description here

GauGau
  • 13
  • 4
  • Provide the HTML... Maybe try to wait for element presence, and after that implement clickable. – Gaj Julije Dec 06 '20 at 09:58
  • @GajJulije thanks for replying me. I also tried it. I even tried to wait for element (that I need) visibility, crawl it, and then wait for button clickable to click it. It still doesn't work. However, Im not sure what parts of HTML I can caption to show here? – GauGau Dec 06 '20 at 11:06

1 Answers1

0

StaleElementReferenceException is thrown when an element is no longer attached to the page. My guess is that sequence looks like:

  1. element has been found
  2. html dom was rebuild - element can be found with used locator BUT it's not the element found in step 1)
  3. js script tries to interact with element found in step 1) BUT it's not THE SAME element (it has the same attributes etc. but it is a different element in case of Selenium

You can try to verify my hypothesis by checking what DOM elements has been added/removed with https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver

Piotr M.
  • 385
  • 2
  • 8