3

I've written a script in python with selenium to traverse different pages staring from the first page through pagination. However, there are no options for next page button except for some numbers. When i click on that number it takes me to the next page. Anyways, when i try to do that with my script, it does click on the second page and goes there but it doesn't slide anymore, I meant instead of going on to the third page it breaks throwing the following error.

line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

Script I'm trying with:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("http://www.cptu.gov.bd/AwardNotices.aspx")
wait = WebDriverWait(driver, 10)
driver.find_element_by_id("imgbtnSearch").click()
for item in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "#dgAwards > tbody > tr > td > a"))):
    item.click()
driver.quit()

Elements within which the pagination numbers are:

<tr align="right" valign="top" style="font-size:XX-Small;font-weight:normal;white-space:nowrap;">
        <td colspan="8"><span>Page: </span><a href="javascript:__doPostBack('dgAwards$ctl01$ctl01','')">1</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl02','')">2</a>&nbsp;<span>3</span>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl04','')">4</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl05','')">5</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl06','')">6</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl07','')">7</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl08','')">8</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl09','')">9</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl10','')">10</a>&nbsp;<a href="javascript:__doPostBack('dgAwards$ctl01$ctl11','')">...</a></td>
    </tr>

Btw, pagination option appears upon clicking on "search" button in the main page.

SIM
  • 21,997
  • 5
  • 37
  • 109

1 Answers1

3

You cannot iterate through the list of pre-defined elements because after you make a click() page refreshes and those elements become stale

You can try below:

from selenium.common.exceptions import NoSuchElementException    

page_counter = 2
while True:
    try:
        if not page_counter % 10 == 1:
            driver.find_element_by_link_text(str(page_counter)).click()
            page_counter += 1
        else:
           driver.find_elements_by_link_text("...")[-1].click() 
           page_counter += 1
    except NoSuchElementException :
        break

This should allow you to switch to next page while it's possible

Andersson
  • 51,635
  • 17
  • 77
  • 129
  • You are the wizard sir, Andersson. This is the best technique I've ever come across. – SIM Aug 14 '17 at 20:34
  • One thing sir- why the percentage "page_counter % 10"? Forgive my ignorance. – SIM Aug 14 '17 at 20:41
  • 1
    In this content it's not a *percentage*, but [*modulo*](https://stackoverflow.com/questions/12754680/modulo-operator-in-python). `page_counter % 10 == 1` means to return `page_counter` that equals to `10 + 1` (`11`), `20 + 1` (`21`), `30 + 1` (`31`)... That's because instead of links with text `11`, `21`, `31`... there are links with `...` – Andersson Aug 14 '17 at 20:47
  • You made my day, sir. Thankssssss a trillion. – SIM Aug 14 '17 at 20:51
  • Please take here a look sir Andersson, "https://stackoverflow.com/questions/45754132/webdriver-is-unable-to-traverse-by-clicking-on-the-next-page-button" – SIM Aug 18 '17 at 10:35