How to click on the 'Show More' button on a lazy loading webpage

Question

I'm trying to click 'Show More' button on a web page. I wrote this code but get an error below the code.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait

#Open Chrome
driver=webdriver.Chrome(executable_path="C:/Users/chromedriver.exe")

web="https://www.expedia.com/Hotel-Search?adults=1&destination=Montreal%2C%20Quebec%2C%20Canada&endDate=2019-09-16&latLong=45.50195%2C-73.56714&localDateFormat=M%2Fd%2Fyyyy&regionId=178288&sort=recommended&startDate=2019-09-15&useRewards=true"
driver.get(web)
driver.maximize_window()

#parse html        
html =driver.page_source
soup=BeautifulSoup(html,"html.parser")
time.sleep(5)
WebDriverWait(driver, 5)

#click show more
show_more=driver.find_element_by_link_text('Show More')
#Another element is covering the element you are to click.    
driver.execute_script("arguments[0].click();", show_more)

Error:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Show More"}

At first, I didn't include the last line (execute_script...) but thought it would work if I include that line but still the same.

Any help would be appreciated.

An extra question is that is there any way to click 'Show More' button multiple times? because I noticed that I had to click multiple times to look up all the hotel listings.

score 1 · Accepted Answer · answered Sep 09 '19 at 05:50

1

I would simply use the below css.

button[data-stid='show-more-results']

Script:

# scroll to the element before clicking
driver.find_element_by_css_selector("button[data-stid='show-more-results']").location_once_scrolled_into_view
# click on the show more button
driver.find_element_by_css_selector("button[data-stid='show-more-results']").click()

Screenshot:

answered Sep 09 '19 at 05:50

supputuri

13,644
2
21
39

Since you are using the same locator more than once, you should store it in a variable and then use the variable in the two finds. That way if the locator changes, you only have to change it in one spot. – JeffC Sep 09 '19 at 19:19

score 0 · Answer 2 · answered Sep 09 '19 at 07:21

You need to click on the button with text as Show More multiple times after scrolling it within the Viewport and to achieve that you can use the following Locator Strategies:

Code Block:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
chrome_options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.expedia.com/Hotel-Search?adults=1&destination=Montreal%2C%20Quebec%2C%20Canada&endDate=2019-09-16&latLong=45.50195%2C-73.56714&localDateFormat=M%2Fd%2Fyyyy&regionId=178288&sort=recommended&startDate=2019-09-15&useRewards=true")
while True:
    try:
        driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH, "//span[text()='Show More']"))))
        driver.execute_script("arguments[0].click();", WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Show More']"))))
        print("Show More button clicked")
    except:
        print("No more Show More button")
        break
driver.quit()

Console Output:

Show More button clicked
Show More button clicked
Show More button clicked
.
No more Show More button

score 0 · Answer 3 · answered Sep 11 '19 at 14:14

0

Try: driver.findElement(By.xpath("//span[contains(text(),'Show More')]")).click()

answered Sep 11 '19 at 14:14

Clever Brains

1

How to click on the 'Show More' button on a lazy loading webpage

3 Answers3