ElementNotVisibleException: Message: element not interactable error while trying to click on the top video in a youtube search

Question

I cannot seem to find a way to click on the right element in order to get the url I am looking for. In essence I am trying to click on the top video in a youtube search (the most highly ranked returned video).

How to resolve ElementNotInteractableException: Element is not visible in Selenium webdriver? -> This is for Java but it let me in the right direction (knowing I needed to execute JavaScript)

http://www.teachmeselenium.com/2018/04/17/python-selenium-interacting-with-the-browser-executing-javascript-through-javascriptexecutor/ -> This shows me how I should try to execute the javascript in python.

I have also seen countless articles about waits but they do not solve my problem.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

wrds = ["Vivaldi four seasons", "The Beatles twist and shout", "50 
cent heat"] #Random list of songs

driver = webdriver.Chrome()

for i in wrds:
    driver.get("http://www.youtube.com")
    elem = driver.find_element_by_id("search")
    elem.send_keys(i)
    elem.send_keys(Keys.RETURN)

    time.sleep(5)
    driver.execute_script("arguments[0].click()",driver.find_element_by_id('video-title')) #THIS CLICKS ON WRONG VIDEO
    #elem = driver.find_element_by_id("video-title").click() #THIS FAILS
    time.sleep(5)

    url = driver.current_url

driver.close()

I get a ElementNotVisibleException: Message: element not interactable error when I do not execute any javascript (even though it has actually worked before it is just no way near robust). When I do execute the javascript it clicks on the wrong videos.

I have tried all types of waits "Explicit" and "Implicit" this did now work. I am quite sure I need to execute some JavaScript but I don't know how.

Answer 1

You were almost there. You need to induce WebDriverWait for the element to be clickable and you can use the following solution:

• Code Block:

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  wrds = ["Vivaldi four seasons", "The Beatles twist and shout", "50 cent heat"]
  kwrd = ["Vivaldi", "Beatles", "50"]
  options = webdriver.ChromeOptions()
  options.add_argument("start-maximized")
  options.add_argument("disable-infobars")
  options.add_argument("--disable-extensions")
  driver=webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\\chromedriver.exe')
  for i, j in zip(wrds, kwrd):
      driver.get("https://www.youtube.com/")
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#search"))).send_keys(i)
      driver.find_element_by_css_selector("button.style-scope.ytd-searchbox#search-icon-legacy").click()
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "h3.title-and-badge.style-scope.ytd-video-renderer a"))).click()
      WebDriverWait(driver, 10).until(EC.title_contains(j))
      print(driver.current_url)
  driver.quit()
  

Answer 2

That's one of the reasons you should never use JavaScript click, Selenium Webdriver is designed to stimulate as if a real user is able to click. Real user can't click an invisible element in the page but you can click through Javascript. If you search the element by that id video-title, it matches totally 53 videos. But I don't know which one you want to click. You may match that element by some other way(not by id).

I will give you an idea how to click that element but you need to find out the index first before you click.

driver.find_element_by_xpath("(//*[@id='video-title'])[1]").click

If the first one is invisible, then pass 2, [2] then three, figure out which one it's clicking the desired element. Or you may specify the exact element, we may try to locate that element by some other way.