0

I'm trying to code a python script using selenium to automatically pick something randomly for dinner by using the inputted location. However, I've been getting these error messages: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="tsuid11"]/div[2]/div/a/div/div[3]/div"}

I don't really understand why this error is happening. I've even watched the entire process load multiple times and am also certain the XPath value is correct.

This is my code:

import requests
import random
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

location = input("Please enter your postal code: ")

driver = webdriver.Chrome("<path to chromedriver.exe>")

query = "food near " + location
print("Please give us a moment...")
driver.get("https://www.google.com/search?q=" + query)
time.sleep(3)

#this helps to click the "View All" option to see the entire list of restaurants nearby
view_all = driver.find_element_by_xpath('//*[@id="rso"]/div[1]/div/div/div/div/div[5]/div/g-more-link/a/div/span[1]')
view_all.click()
time.sleep(10)

#This is where I can't seem to find the element by its XPath
name = driver.find_element_by_xpath('//*[@id="tsuid11"]/div[2]/div/a/div/div[3]/div')
print(name)

driver.close()

I have also already searched this error up and got this: here

Based on the answer, the person mentioned "Could be a race condition where the find element is executing before it is present on the page". However, I've already added the time.sleep() function to mitigate that.

Any help would be appreciated :)

Update: I got it to work by replacing find by XPath to find by CSS Selector. However, this is just a work around, I'm still going to try to figure this one out. Thanks for all the solutions, but unfortunately none of them worked for me.

  • my View All button works perfectly fine on my end
  • the webdriver wait function also didn't allow me to find the element (I think time.sleep does the exact same thing)
  • and the element is not inside an iframe

After some further probing, I've tested the full XPath: /html/body/div[6]/div/div[7]/div[1]/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[1]/div[4]/div[3]/div[2]/div/a/div/div[3]/div on the browser console itself, and did not find anything. By digging through the layers, I've noticed that it stops at /html/body/div[6]/div/div[7]/div[1]/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[1]/div[4]/div[3]/div[2]/div, right before the a tag. I'm not sure why as of now, will update again if I find anything.

Update 2: I used class name instead of XPath which is much more consistent for my output. That fixed everything for me :) Hope this helps.

CBCH
  • 127
  • 1
  • 3
  • 14
  • What exactly you wanna do after clicking on view all ? Which css you are using ? – cruisepandey Jul 27 '21 at 06:28
  • I'm getting the name of the restaurant(s) and print them out on the console for now. Sorry for the unclarity. – CBCH Jul 27 '21 at 06:31
  • Did you try out my solution below ? It should work fine, let me know if you face any issue – cruisepandey Jul 27 '21 at 06:32
  • Yep, it worked by CSS Selector, but just for learning purposes, I'm trying to figure out why my XPath stops at a certain layer for some reason. – CBCH Jul 27 '21 at 06:37
  • no, your xpath is absolute, so it's gonna be inconsistent. You can try with relative xpath. – cruisepandey Jul 27 '21 at 06:38
  • Yep, you are right about it being inconsistent. However, I've manually checked for it through the automated browser when the XPath is correct. However, I can't find it unless I go down into a certain directory. – CBCH Jul 27 '21 at 06:39
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/235324/discussion-between-cbch-and-cruisepandey). – CBCH Jul 27 '21 at 06:40
  • I have given this xpath `div[role='heading'] div`, which is pretty simple from readability perspective and works like a charm. you can have equivalent xpath for the same. but why ? CSS are preferable than xpath. – cruisepandey Jul 27 '21 at 06:40

4 Answers4

1

After clicking on view All, you have not mentioned what exactly you wanna do, if I assume that you wanna fetch restaurant name, you could do that by the below method :

you can write below code after these two lines from your code : -

view_all.click()
time.sleep(10)

Sample code :

all_names = driver.find_elements(By.CSS_SELECTOR, "div[role='heading'] div")
print(all_names[0].text)

or in case you would like to fetch all the names :-

names = []
for name in driver.find_elements(By.CSS_SELECTOR, "div[role='heading'] div"):
    names.append(name.text)


print(names)

Updated 1 :

driver = webdriver.Chrome("<path to chromedriver.exe>")
driver.maximize_window()
driver.implicitly_wait(30)
location = "545084"
query = "food near " + location
driver.get("https://www.google.com/search?q=" + query)
wait = WebDriverWait(driver, 10)
ActionChains(driver).move_to_element(wait.until(EC.element_to_be_clickable((By.XPATH, "//a[contains(@href, '/search?tbs')]")))).perform()
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[contains(@href, '/search?tbs')]"))).click()
names = []
for name in driver.find_elements(By.XPATH, "//div[@aria-level='3']"):
names.append(name.text)

print(names)

Output :

["Mum's Kitchen\nCatering Pte Ltd", 'Kitch', "Zoey's Diner", "McDonald's", 'ThaiExpress', 'Dian Xiao Er', "Swensen's", 'Pho Street Compass One', 'Singapore WaterDrop Tea House', 'LeNu at Compass One', "McDonald's Compass One", "McDonald's", 'Miami Bistro', 'Monster Curry', 'Texas Chicken (Hougang Capeview)', 'Paradise Hotpot at Compass One', "Long John Silver's Rivervale Mall", 'Boon Tong Kee @ Compass One', 'Din Tai Fung', 'Fish & Co.', 'PUTIEN', 'Soup Restaurant 三盅两件 - Compass One']
cruisepandey
  • 28,520
  • 6
  • 20
  • 38
0

Please check this element is inside the frame, when we use the debugger, our element would be identified but the code will not know. we need to drill down from root frame -> corresponding frame -> then the element would be intractable.

Jayanth Bala
  • 758
  • 1
  • 5
  • 11
0

#Change your xpath to click on view all

view_all = driver.find_element_by_xpath('//*[@id="rso"]//span[normalize-space(text())="View all"]')

#This will print all restaurants name

numberof_restaurants = driver.find_elements_by_xpath('//*[contains(@id,"tsuid")]//div[@role="heading"]')
for restaurant in numberof_restaurants:
    name = restaurant.text
    print(name)
cruisepandey
  • 28,520
  • 6
  • 20
  • 38
Ashok Kakade
  • 51
  • 5
0

Can you try the below lines of codes, I hope it will help

location = input("Please enter your postal code: ")

driver = webdriver.Chrome("<path to chromedriver.exe>")
#I've try to open the browser in the maximize mode because when we try to click on the `veiw All` button sometime it through the Exception
driver.maximize_window()
query = "food near " + location
print("Please give us a moment...")
driver.get("https://www.google.com/search?q=" + query)
time.sleep(3)

#This XPath also gives me an error sometime, So I've updated it to click on restaurants nearby
view_all = driver.find_element_by_xpath("//*[text()='View all']")
view_all.click()
time.sleep(10)

#the provided xpath was not perfect I was not able to search it on the page, Also updated the print() with `.text` to get the text
name = driver.find_element_by_xpath("//div[@id='tsuid9']//a")
print(name.text)

driver.close()

You can try to, click on the element with explicitWait

viewAll = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//*[text()='Viewall']")))
viewAll.click()

import

import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

O/P enter image description here

YaDav MaNish
  • 1,260
  • 2
  • 12
  • 20