How to click load more button in selenium untill page endspython?

Question

Using selenium to load and page and need to click load more button, but couldn't able to do that.

Tried this :

from selenium import webdriver
import pandas as pd
driver = webdriver.Chrome('/Users/1/chromedriver.exe')
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')

driver.find_element_by_css_selector(".btn.btn-primary.btn-lg").click();

tried the above but button is clicking and there are multiple load more how to load them multiple times untill page gets over

Error :

Tried to keep it in loop but getting :

element not interactable

`.click()`, you're not calling the method. Don't forget `()`s — abdusco, Jul 15 '19 at 07:20
button is clicking and there are multiple rows and how to load them multiple times untill page gets over — , Jul 15 '19 at 07:34

score 1 · Answer 1 · answered Jun 10 '20 at 02:36

The solution that worked for me is a little simple and requires a little effort but works fine.

count=20
while count>1:
    button=driver.find_element_by_css_selector("button.ipl-load-more__button")
    button.click()
    count-=1
    time.sleep(2)
//do you work once all the pages are loaded

The only thing you need to worry about is setting the right count value, if its too small you might get thrown an error, just catch it and increase/decrease the count value according to your requirement. I hope this helps.

undetected Selenium · Answer 2 · 2019-07-15T12:38:45.497

To click() on the element with text as LOAD MORE RESULTS you need to induce WebDriverWait for the desired element_to_be_clickable() and you can use the following Locator Strategies:

Code Block A:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions() 
chrome_options.add_argument("start-maximized")
# chrome_options.add_argument('disable-infobars')
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://simpletire.com/catalog?select=1&brand=61&query=catalog")
while True:
    try:
        # WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='btn btn-primary btn-lg']//span[@class='glyphicon glyphicon-play']"))).click()
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(., 'Load More Results')]"))).click()
        print("LOAD MORE RESULTS button clicked")
    except TimeoutException:
        print("No more LOAD MORE RESULTS button to be clicked")
        break
driver.quit()

Code Block B:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

chrome_options = webdriver.ChromeOptions() 
chrome_options.add_argument("start-maximized")
# chrome_options.add_argument('disable-infobars')
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://simpletire.com/catalog?select=1&brand=61&query=catalog")
while True:
    try:
        # ActionChains(driver).move_to_element(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='btn btn-primary btn-lg']//span[@class='glyphicon glyphicon-play']")))).pause(3).click().perform()
        ActionChains(driver).move_to_element(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(., 'Load More Results')]")))).pause(5).click().perform()
        print("LOAD MORE RESULTS button clicked")
    except TimeoutException:
        print("No more LOAD MORE RESULTS button to be clicked")
        break
driver.quit()

Console Output:

LOAD MORE RESULTS button clicked
LOAD MORE RESULTS button clicked
LOAD MORE RESULTS button clicked
LOAD MORE RESULTS button clicked
LOAD MORE RESULTS button clicked
.
.
.

Browser Snapshot:

`ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (847, 918). Other element would receive the click: — , Jul 15 '19 at 11:09
`C:\Users\1\Anaconda3\lib\site-packages\ipykernel_launcher.py:16: DeprecationWarning: use options instead of chrome_options app.launch_new_instance() ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (847, 918). Other element would receive the click: — , Jul 15 '19 at 11:20
@thoris Try the second code block and let me know the status. — undetected Selenium, Jul 15 '19 at 11:52
@thoris Updated the second code block. Let me know the status. — undetected Selenium, Jul 15 '19 at 12:03
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/196470/discussion-between-thor-is-and-debanjanb). — , Jul 15 '19 at 12:05

score 0 · Accepted Answer · answered Jul 25 '19 at 21:13

Here is the code that should work. I am not sure how many tires available, the script ran successfully to load ~1000 results.

I have given the option to stop loading after meeting tires count, rather iterating 100+ times.

url = 'https://simpletire.com/catalog?select=1&brand=61&query=catalog'
driver.get(url)
loadingButton = WebDriverWait(driver,30).until(EC.presence_of_element_located((By.XPATH,"//div[@id='load_button']")))
maxTires = 200;
while loadingButton:
    loadingButton.click()
    time.sleep(2)
    WebDriverWait(driver,30).until(EC.presence_of_element_located((By.XPATH,"//div[@id='is_loading'][contains(@style,'none')]")))
    loadElems = driver.find_elements_by_xpath("//div[@id='load_button'][contains(@style,'block')]")
    if len(loadElems)>0:
        loadingButton = driver.find_element_by_xpath("//div[@id='load_button'][contains(@style,'block')]")
        tiresLoaded = len(driver.find_elements_by_css_selector(".catResultWrapper.result"))
    else:
        print("Loaded all the tires")
        break
    if tiresLoaded >= maxTires:
        print (tiresLoaded + " are loaded successfully.")
        break

How to click load more button in selenium untill page endspython?

3 Answers3