0

Currently using Selenium+ Python and this code: 

browser.get('http://www.google.com')
search = browser.find_element_by_name('q')
search.send_keys("how to search the internet")
search.send_keys(Keys.RETURN) # hit return after you enter search text
time.sleep(5) # sleep for 5 seconds so you can see the results
browser.execute_script("window.scrollTo(0, 873)") 
time.sleep(2)
browser.find_element(By.XPATH, '(//h3)[3]/a').click()`

as my code.

What Happens: Goes to Google and types in the words and hits search. Scrolls down a bit and then it clicks the first link

What I want: I want to be able to click a random link from the search result page

How to do this?

Edit: This is what I meant when I said unnecessary: Unnecessary 1: https://i.stack.imgur.com/HBzFg.jpg

Unnecessary 2: https://i.stack.imgur.com/j4xGq.jpg

These are the only results I want: This: https://i.stack.imgur.com/lAdHA.jpg

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Kunwar Sodhi
  • 213
  • 2
  • 5
  • 13
  • 1
    You'll need to collect the links from the page and then make a random choice from that list. See the `random` package documentation for the `choice` method. – Prune Aug 22 '18 at 22:57
  • Thanks for the help. I figured out how to do collect all the links from the page. However there are some links that are unnecessary. Is it possible to exclude them? – Kunwar Sodhi Aug 23 '18 at 02:39
  • If you got *unnecessary links* then your selector is not perfect. Use more specific selector to get rid of them – Andersson Aug 23 '18 at 04:58
  • Hi @KunwarSodhi, what exactly do you mean by unnecessary links? Do you want to edit your post and give us an example of what exactly you need - which links exactly, and we could provide a detailed solution. – Shlomi Bazel Aug 23 '18 at 08:21

1 Answers1

1

As per your question to click() on a random link from google search results, as per your code trial if you invoke window.scrollTo(0, 873) and then invoke click() as in:

find_element(By.XPATH, '(//h3)[3]/a').click()`

Selenium will still try to attempt click() on the first match, which may not be your desired usecase.

Solution

Inorder to click() on a random link from google search results, you can create a List out of the search results and then generate a Random Number and invoke click() through an index as follows:

  • Code Block:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys 
from random import randint

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_argument('disable-infobars')
browser=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
browser.get('http://www.google.com')
search = browser.find_element_by_name('q')
search.send_keys("selenium")
search.send_keys(Keys.RETURN)
my_search_list = WebDriverWait(browser, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//h3[@class='r']/a[not(ancestor::div[@class='xIleA'])]")))
myRandomNumber = randint(0, len(my_search_list))
print(myRandomNumber)
my_search_list[myRandomNumber].click()

  • Console Output:

    4
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Can you ensure you are accessing `http://www.google.com` as your program might be accessing the local google server instance which is probhably not ending with `.com`. In that case you may require to tweak the _xpath_ but the logic remains the same. In this run I got an output as **6**. – undetected Selenium Aug 23 '18 at 14:30
  • It works!. In this case I got a output of 2. However the first time I ran I got a output of 10 and it didn't work then. Maybe the 10th link isn't on the webpage at the time? – Kunwar Sodhi Aug 23 '18 at 14:36
  • Yup I just got 10 as a output and it didn't work and said List Index out of range – Kunwar Sodhi Aug 23 '18 at 14:37
  • Well, till now I got a maximum of **9** only :) but why would you get _List Index out of range_ ? We haven't hard coded any value ... – undetected Selenium Aug 23 '18 at 14:38
  • https://imgur.com/a/IgFlTJN I have no idea why I'm getting this error – Kunwar Sodhi Aug 23 '18 at 14:49
  • Hmmm, _Failed to create shader cache entry_ I suppose is a much finer issue with your system os/chores. See this discussion: [Failed to create shader cache entry- error while locating an element by its Css selector](https://stackoverflow.com/questions/46393818/failed-to-create-shader-cache-entry-error-while-locating-an-element-by-its-css) and the linked discussions. – undetected Selenium Aug 23 '18 at 15:00
  • Yaa I tried to fix that error. But I am talking about Line 19 and list out of index. Does the shader error have anything to do with this? – Kunwar Sodhi Aug 23 '18 at 15:01
  • I am pretty much confused by the error as we haven't hardcoded any size for the _List_ as such :( I have executed almost 20 times without any error. Did you `from random import randint`? – undetected Selenium Aug 23 '18 at 15:04
  • Yup. I think its because the webpage doesn't have more then 10 results? – Kunwar Sodhi Aug 23 '18 at 15:10
  • Ya it only happens when the code RandomNumber is = to 10. All the other numbers work fine – Kunwar Sodhi Aug 23 '18 at 15:13
  • See `myRandomNumber` is calculated from the _List_ length, so `my_search_list[myRandomNumber]` can't be **out of range** anyway. – undetected Selenium Aug 23 '18 at 15:15
  • I tried with searching for youtube instead of selenium. So far it has not gotten to using 10.I think it was a problem with the search? – Kunwar Sodhi Aug 23 '18 at 15:26
  • Exactly, I even didn't cross 9 for once. – undetected Selenium Aug 23 '18 at 15:28