0

I'm having a lot of issues understanding how to do this. What i need to do is simple, which is to flag whenever my automated google search is not able to find any search results. My code example:

driver = webdriver.Chrome(executable_path)
driver.get("https://google.com/")
search = driver.find_element_by_name("q")
search.send_keys('site:'+'www.pa.gov'+ ' "ADT.com" '+'\n')
if driver.find_element(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()[2]"):
    print(True)
else:
    print(False)

I keep getting this error:

InvalidSelectorException: invalid selector: The result of the xpath expression "//*[@id='topstuff']/div/div/p[1]/text()[2]" is: [object Text]. It should be an element.
  (Session info: chrome=87.0.4280.88)

This is the link i've searched No Results

What am I doing wrong?

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Nate
  • 136
  • 10
  • Selenium doesn't support `text()` node. You need to stop at `//*[@id='topstuff']/div/div/p[1]` and get the entire text. And after extract your needed text. Or find another `xpath` which not use `text()` as node. – KunLun Dec 15 '20 at 18:57

2 Answers2

0

This error message...

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "//a[following-sibling::input[@value="ST"]]/@href" is: [object Attr]. It should be an element.

......implies that your XPath expression was not a valid xpath expression.

You need to replace:

driver.find_element(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()[2]")

with:

driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()")[2]

Additional Considerations

Selenium WebDriver supports xpath-1.0 only which returns the set of node(s) selected by the xpath.

You can find the specifications in XML Path Language (XPath) Version 1.0

However the xpath expression:

driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()")[2]

Is a xpath-2.0 based expression and would typically return an object Text. A couple of examples:

  • //@version: selects all the version attribute nodes that are in the same document as the context node
  • ../@lang: selects the lang attribute of the parent of the context node

You can find the specifications in XML Path Language (XPath) 2.0 (Second Edition)

Solution

So effectively your block of code will be:

if driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]")[2]:
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I'm getting this error now: a= driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]")[2] IndexError: list index out of range – Nate Dec 15 '20 at 18:10
  • @NateLeMay You haven't provided us the HTML of the element. So it's impossible to help you further with `IndexError: list index out of range` as we can't assume how many elements were identified and placed within the list but definitely this solution addresses your initial error of **InvalidSelectorException: invalid selector** – undetected Selenium Dec 15 '20 at 18:17
  • Selenium doesn't support node `text()` – KunLun Dec 15 '20 at 19:01
  • @KunLun Where did you find `text()` in my effective/proposed solution? – undetected Selenium Dec 15 '20 at 19:02
  • @KunLun Care to answer my counter question as per your comment above? Where did you find `text()` in my effective/proposed solution? – undetected Selenium Dec 15 '20 at 19:11
  • Off: Please have some patience. I'm not here all the time. On: `driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()")[2]` - this xpath returns `Text` node type. Selenium support only `Element` type. – KunLun Dec 15 '20 at 22:09
  • @KunLun That wasn't my question either. Where did you find `text()` in my _**effective**_ / _**proposed**_ (_read the previous word carefully_) solution? – undetected Selenium Dec 15 '20 at 22:20
  • `You need to replace: ... with: driver.find_elements(By.XPATH, "//*[@id='topstuff']/div/div/p[1]/text()")[2]` . Also your effective solution `xpath` select 2nd `//*[@id='topstuff']/div/div/p[1]` Element. OP wants to get 2nd text node from `//*[@id='topstuff']/div/div/p[1]` – KunLun Dec 16 '20 at 12:16
  • @KunLun Definitely there is an issue with OP's xpath `//*[@id='topstuff']/div/div/p[1]/text()[2]` and being a valuable contributor you don't just publish code block as answers. To serve the broader audience you need to explain all the issues one by one. – undetected Selenium Dec 16 '20 at 12:25
-1

Xpath you have provided doesn't supported by selenium yet.

I believe you are capturing the error for wrong website.

Induce WebDriverWait() and wait for visibility_of_element_located() and following xpath.

Induce try..except block to handle if any error occurs.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver=webdriver.Chrome(executable_path)
driver.get("https://google.com/")
search = driver.find_element_by_name("q")
search.send_keys('site:'+'www.pa.gov'+ ' "ADT.com" '+'\n')
try:
  Searchelement=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH, "//div[@id='topstuff']//p[@role='heading']")))
  print(True)
  print("============================")
  print(Searchelement.text)
  print("============================")
  #If you want to get specific node value then try below xpath
  print(driver.find_element_by_xpath("//div[@id='topstuff']//p[@role='heading']/span/em").text)
except:
  print(False)

Console output:

True
============================
Your search - site:www.pa.gov "ADT.com" - did not match any documents.
============================
site:www.pa.gov "ADT.com"
KunduK
  • 32,888
  • 5
  • 17
  • 41