4

so I am doing some web scraping using Selenium with Python and I am having a problem. I am clicking a Next button to move to the next page on a certain website, but I need to stop clicking it when I reach the last page. Now, my idea of doing it would be just to use some_element.click() in a try/except statement and wait until it gives me an error while the button is not clickable anymore. It seems though, .click() doesn't emit any signal of any kind, it doesn't throw an error when the button can't be clicked and it doesn't emit any true or false signal.

a code snippet I tried using:

while True:
    try:
       button = driver.find_element_by_class_name('Next_button')
       button.click()
    except:
       break

Is there any other way? Thanks and cheers.

Ivan Bilan
  • 2,379
  • 5
  • 38
  • 58
  • Is the element displayed when you reach the last page? – Saifur Jun 07 '15 at 19:01
  • 1
    It could be that the link is still clickable, but instead of moving to the next page, it does nothing. Can you share the link to the website you are running the code against? – alecxe Jun 07 '15 at 19:02
  • @Saifur is right. Check what html attributes change for that button when you reac the last page. – Vikas Ojha Jun 07 '15 at 19:02
  • Saifur, the Next page button is still shown, but it is greyed out on the last page. @Vikas Neha Ojha, the class name changes at the last page and so does the xpath to it. How can I detect this change in Python? Or did I understand your advice wrongly. alecxe, yes, that could be true. The loop just goes on forever. – Ivan Bilan Jun 07 '15 at 19:16
  • 1
    The loop just goes on forever - Please mention whether it is even navigating through the next webpages. You mentioned that class name changes and I can see that you are trying to find the element by class name only, so still your code should work. Could you please post the html of just the button on initial page and the last page? – Vikas Ojha Jun 07 '15 at 19:19
  • I can see the browser going through each of the page and saving the content of each page until the last one. At the last page, the loop is never broken, the script just keeps running. The code for the next page button on the page before the last one:
  • On last page: – Ivan Bilan Jun 07 '15 at 19:33