Can't find and click on element on webpage

Question

I am trying to get Selenium to click on "see more hours" and grab the hours shown on the next page on Google Maps. My issue is performing the first step: clicking.

What I have tried:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup
import time



url = 'https://www.google.com/maps/place/211+Clover+Lane/@38.2567237,-85.6849465,13z/data=!4m10!1m2!2m1!1srestuarant!3m6!1s0x886974dd922d909f:0x9c4b766c51b27adc!8m2!3d38.2567237!4d-85.6499276!15sCgpyZXN0dWFyYW50WgwiCnJlc3R1YXJhbnSSARNldXJvcGVhbl9yZXN0YXVyYW504AEA!16s%2Fg%2F1tcvcn03?entry=ttu'

options = Options()

driver = webdriver.Chrome(executable_path=ChromeDriverManager(log_level=0).install(), options=options)
driver.get(url)
time.sleep(1)
response_overview = BeautifulSoup(driver.page_source, 'html.parser')


try:
    wait = WebDriverWait(driver, 10)
    # hours_bt = wait.until(EC.element_to_be_clickable((By.XPATH, "//button[contains(@aria-label, 'See more hours')]")))
    hours_bt = driver.find_element_by_xpath("//button[contains(@aria-label, 'Close')]")
    print(hours_bt.text)
    hours_bt.click()
    time.sleep(1)         
    hours_page =  BeautifulSoup(driver.page_source, 'html.parser')
    hours = hours_page.find('div', class_='t39EBf GUrTXd')['aria-label'].replace('\u202f', ' ') 
        
except Exception as e1: 
    print('e1', e1)
    try:

        hours_bt = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="QA0Szd"]/div/div/div[1]/div[2]/div/div[1]/div/div/div[11]/div[4]/button')))
        hours_bt.click()
        time.sleep(1)         
        hours_page =  BeautifulSoup(driver.page_source, 'html.parser')
        hours = hours_page.find('div', class_='t39EBf GUrTXd')['aria-label'].replace('\u202f', ' ')   
        
    except Exception as e2:
        hours = None
        print('e2', e2)
        pass

The first try says that the element is not interactable and the second try times out. I'm at my wit's ends and appreciate some help. Versions: selenium==3.14.0,

Have you tried targeting css element `div[role="button"][jsaction*=".openhours."]`? I think the one with XPath `//button[contains(@aria-label, 'Close')]` becomes visible after opening that - and is probably not the tag you want — Driftr95, Jul 28 '23 at 22:22

score 0 · Answer 1 · answered Jul 28 '23 at 20:32

0

I ran the same code using Firefox instead of Chrome. When the driver tried to get the url I was redirected to this page:

If you're having the same issue, you can run this line of code to click the "Accept all" button. It should automatically re-direct the browser to the correct url afterwards.

driver.find_element_by_xpath("//button[contains(@aria-label, 'Accept all')]").click()

answered Jul 28 '23 at 20:32

Kenneth Granahan

47
3

Thank you but I do have this step in my code already and this is not the cause of the issue for me – Saeed Jul 28 '23 at 20:52

score 0 · Accepted Answer · answered Jul 28 '23 at 23:19

To click on the element with text as See more hours you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label$='See more hours'][data-item-id='oh']"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(@aria-label, 'See more hours')][@data-item-id='oh']"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

score 0 · Answer 3 · answered Aug 11 '23 at 11:00

0

you can use GoogleMapsPy and scrape data from google maps without browser need, you can find days in data

answered Aug 11 '23 at 11:00

ammar alkotb

194
6

Thank you. Can you elaborate how this can work with neither API or screen scraping with a browser? – Saeed Aug 11 '23 at 22:52
It simulates a browser, show https://github.com/3mora2/GoogleMapsPy/blob/475336f0a72f1e38fdc3ccda9463e2aa6e64a0a2/GoogleMapspy/google_maps.py#L231 – ammar alkotb Aug 12 '23 at 07:13

Can't find and click on element on webpage

3 Answers3