0

I'm aiming to get URLs as a results for my search_term. By running the below code in Python, my only output is []. Would someone be able to help to modify the code in order to receive the list with URLs to search results, ideally limited to first 10-20 URLs? Thanks in advance, please find the code below:

import selenium.webdriver as webdriver

def get_results(search_term):
    url = "https://www.google.com"
    browser = webdriver.Safari()
    browser.get(url)
    search_box = browser.find_element_by_name("q")
    search_box.send_keys(search_term)
    search_box.submit()
    try:
        links = browser.find_elements_by_xpath("//ol[@class='web_regular_results']//h3//a")
    except: 
        links = browser.find_elements_by_xpath("//h3//a")
    results = []
    for link in links:
        href = link.get_attribute("href")
        print(href)
        results.append(href)
    browser.close()
    return results

get_results("fish")
Baobab1988
  • 685
  • 13
  • 33

1 Answers1

0

You have provided wrong xpath that is why you are not getting it.I have tried with chrome and change the xpath it is working fine.Try this and let know.

def get_results(search_term):
url = "https://www.google.com"
browser = webdriver.Safari()
browser.get(url)
search_box = browser.find_element_by_name("q")
search_box.send_keys(search_term)
search_box.submit()
try:
    links = browser.find_elements_by_xpath("//h3[@class='r']/a")
except:
    links = browser.find_elements_by_xpath("//h3[@class='r']/descendant::a")
print(len(links))
results = []
for link in links:
    href = link.get_attribute("href")
    print(href)
    results.append(href)
browser.close()
return results

get_results("fish")

KunduK
  • 32,888
  • 5
  • 17
  • 41
  • Hi Kajal, I'm trying to avoid Chrome, so I used your suggestion above, and replaced webdriver.Chrome() with webdriver.Safari(), but then it returned 0 results. Would you know how to get this to work with Safari? Thanks! – Baobab1988 Feb 09 '19 at 14:47
  • Ok I’ll check and let you know. – KunduK Feb 09 '19 at 15:35
  • I don't have safari browser 10 or above to verify the code I have updated the previous code.Try with safari this time and let me know whether it works. – KunduK Feb 09 '19 at 16:48
  • Hi Kajal, it still shows 0 results when trying with Safari 12. Would there be any other way? Thanks in advance! – Baobab1988 Feb 10 '19 at 09:59
  • Can you share me safari 12 as I have safari 5.1.7 so can’t test and verify.If u could send the safari 12 then I can surely give you some solution.Thanks – KunduK Feb 10 '19 at 10:05
  • Hi! Are you on Windows or Mac? I think 5.1.7 was the latest release for windows and since then there are no updates :( – Baobab1988 Feb 10 '19 at 11:42
  • Yes I am on Windows – KunduK Feb 10 '19 at 13:08