How to click on button with text Show more results using Selenium and Python?

Question

For this button:

I used these pieces of code, but it did not work.

WebDriverWait(wd, 1).until(EC.element_to_be_clickable((By.XPATH, "//input[contains(., 'Show more results')]"))).click()

and

button = wd.find_elements_by_xpath("//*[contains(text(), 'My Button')]")
button.click()

https://www.google.com/search?q=aircraft+fuselage+airplane+fly+aviation+aircraft+wing+aerodynamic+plane+aircraft&rlz=1C1CHBF_enUS985US986&sxsrf=ALiCzsbctupnUvetvOBSuPGjv8vRok0SQg:1658425706014&source=lnms&tbm=isch&sa=X&ved=2ahUKEwiY896YxYr5AhXXpokEHRjuCkEQ_AUoAXoECAEQAw&biw=552&bih=679&dpr=1.25 — Fateme Nazari, Jul 27 '22 at 19:09
Did you also figure out the scrolling, and reaching the end of page? — Barry the Platipus, Jul 27 '22 at 22:29
yes I did it with wd.execute_script('window.scrollTo(0, document.body.scrollHeight);') — Fateme Nazari, Jul 28 '22 at 14:20
That will only scroll the page once. At that point, G Images will load a new set, and you need to scroll it a couple of times as well. How did you account for it? — Barry the Platipus, Jul 28 '22 at 14:25
Someone made an answer here but deleted the answer I got this from him or her. wd.execute_script("var scrollingElement = (document.scrollingElement || document.body);scrollingElement.scrollTop = scrollingElement.scrollHeight;") this seems to be useful. — Fateme Nazari, Jul 28 '22 at 14:31
And how many images did you find in total, for that particular search? Just curious. — Barry the Platipus, Jul 28 '22 at 14:32
There are over 700 images returned by that specific google image search: https://www.google.com/search?q=aircraft+fuselage+airplane+fly+aviation+aircraft+wing+aerodynamic+plane+aircraft&rlz=1C1CHBF_enUS985US986&sxsrf=ALiCzsbctupnUvetvOBSuPGjv8vRok0SQg:1658425706014&source=lnms&tbm=isch&sa=X&ved=2ahUKEwiY896YxYr5AhXXpokEHRjuCkEQ_AUoAXoECAEQAw&biw=552&bih=679&dpr=1.25 — Barry the Platipus, Jul 28 '22 at 14:35
That's what I'm trying to understand: is your code retrieving all of them, clicking the 'load more' button as may times as it appears, and stops when the page says 'Look like you've reach the end' or something like that? — Barry the Platipus, Jul 28 '22 at 14:37
As I said I used some conditions. since I wanted to get the image URL there were some restrictions, for example, I did not want images from Wikipedia and so on. — Fateme Nazari, Jul 28 '22 at 14:38
It's not about that : I'm curious if you managed to click that button as many times as it appeared, scrolled the page as many times as needed to load all of them (dependant on your window size, it can be 4-5 times or more), then stopped once there were no more images? — Barry the Platipus, Jul 28 '22 at 14:40
so sorry for the late response. That's true I could not find any way. — Fateme Nazari, Jul 28 '22 at 17:06

score 1 · Accepted Answer · answered Jul 27 '22 at 20:37

You were almost there. Show more results isn't the innerText but the value of the value attribute.

Solution

To click on the element with text as Show more results you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[value='Show more results']"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@value='Show more results']"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to click on button with text Show more results using Selenium and Python?

1 Answers1

Solution