Selenium - How to know if next page doesn't exist?

Question

I am trying to get all the services title from IBM services page and I am getting this below error:

I want to know if next page exists or not. So, I may break the loop. or keep my loop to that much iterations.

Here is my code:

import time
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException

def writefile(links):
    with open('url_list.txt', 'w') as file:
        file.writelines("%s\n" % link for link in links)

start_url = "https://www.ibm.com/us-en/products/categories?size=30&selectedTopicRoot=technologyTopics&types[0]=service"
links = []
chrome_path = r"C:\Users\IBM_ADMIN\Anaconda3\selenium\webdriver\Chrome\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.get(start_url)
time.sleep(15)

while True:
    time.sleep(5)
    results = driver.find_elements_by_class_name("offering--name")

    for i in range(len(results)):
        links.append(results[i].text)

    writefile(links)

    try:
        next = driver.find_element_by_xpath('//*[@id="IBMAccessibleItemComponents-next"]')
        if (next.is_enabled()):
            next.click()
        else:
            break
    except NoSuchElementException:
        break

driver.close()

Perhaps waiting for the element to be clickable: https://stackoverflow.com/a/28110129/977593 ? — slackmart, Aug 20 '18 at 14:27
@slackmart Nope, I am not waiting for the element to be clickable. I am trynna see if the next page exists or not via seeing if the next button is clickable or not. — Mohammad Zain Abbas, Aug 20 '18 at 14:31

score 2 · Accepted Answer · answered Aug 20 '18 at 14:30

You can use more specific XPath to select Next button only if it is enabled:

try:
    driver.find_element_by_xpath('//*[@id="IBMAccessibleItemComponents-next" and not(@aria-disabled)]').click()
except NoSuchElementException:
    break

Note that on last page aria-disabled="true" added to Next buttons' attributes and so it won't be matched by //*[@id="IBMAccessibleItemComponents-next" and not(@aria-disabled)] XPath

Selenium - How to know if next page doesn't exist?

1 Answers1

Linked