How to send a keyword within the search box and click on search icon on YouTube homepage using Selenium Python

Question

My application is to enter a search keyword into "youtube.com" and check for the results that pop up. I am able to click on the search bar and enter the keyword, but problem is with performing click operation on "search" button. Below is my code:

#!/depot/Python-3.6.2/bin/python
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options
from time import sleep
import sys
import os
import subprocess
options = Options()
options.headless = True
sys.tracebacklimit = 0
options.binary = "/global/freeware/Linux/2.X/firefox-65.0/firefox"
out=subprocess.check_output(['firefox','-v'])
version=out.decode("utf-8").rsplit(".")
version=version[0].rsplit(" ")
if(str(62) < version[-1]):
  gckd_path = "/fs/src/interfaces/cosmos/REGRESSION_PYTHON/Packages/geckodriver"
else:
  gckd_path = "/remote/vgrman/packages/geckodriver_0.18/geckodriver"
arg1=sys.argv[1]
#arg2=sys.argv[2]
driver = webdriver.Firefox(options=options,executable_path=gckd_path)
#url="https://solvnetplus.synopsys.com/s/"
url="https://www.youtube.com/"
print(url)
driver.get(url)
sleep(.3)
username = driver.find_element_by_name("search_query")
username.click()
sleep(.3)
username.send_keys(arg1)
print(username.get_attribute("value"))
password = driver.find_element_by_xpath('//*[@id="search-icon-legacy"]')
print(password.get_attribute("aria-label"))
print(password.get_attribute("id"))
print(password.get_attribute("class"))
password.send_keys(Keys.ENTER)
sleep(5)
print(driver.current_url)
channel = driver.find_elelment_by_xpath('//*[@id="title"]')
print(channel.get_attribute("value"))
print(driver.current_url)
driver.quit()

I did a query to check if I was able to enter keyword into search bar, also querying to check if the search button is found. Click function works fine when I have to enter text into search bar, but does not work when I have to submit my search. Please help me understand what is going wrong here.

Answer 1

EDIT: Youtube looks different in mobile Firefox than on Desktop Chrome. Your question is somehow confusing, are you looking for the instant search results, or are you looking for actual results of the search? I edited the code below to include both scenarios.

The following code will open the page, dismiss the cookie button (if appears), correctly locate and click on the search button, send some keys into the searchbox, locate and printout the instant search results, send Enter to perform the search, and go through results (scroll the page) until it reaches the end of the page It is down to the OP to implement a break from that While loop (it's not hard).

It should all be self explanatory (it also uses Firefox), setup is for Linux, but you just have to observe the imports, and the code after defining the browser.

from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options as Firefox_Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
import time as t

firefox_options = Firefox_Options()

firefox_options.add_argument("--width=1280")
firefox_options.add_argument("--height=720")
firefox_options.set_preference("general.useragent.override", "Mozilla/5.0 (Linux; Android 7.0; SM-A310F Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36 OPR/42.7.2246.114996")

driverService = Service('chromedriver/geckodriver')

browser = webdriver.Firefox(service=driverService, options=firefox_options)

url = 'https://www.youtube.com/'

browser.get(url)

try:
    cookie_button = WebDriverWait(browser, 3).until(EC.element_to_be_clickable((By.XPATH,'//button[@aria-label="Reject the use of cookies and other data for the purposes described"]')))
    print(cookie_button.location_once_scrolled_into_view)
    t.sleep(1)
    cookie_button.click()
    print('rejected cookies')
except Exception as e:
    print('no cookie button')
t.sleep(1)
search_button = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,'//div[@class="mobile-topbar-header-content non-search-mode cbox"]/button[@aria-label="Search YouTube"]/c3-icon')))
search_button.click()
t.sleep(1)
search_field = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,'//input[@name="search"]')))
search_field.send_keys('programming is hard')
t.sleep(1)
results = WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH,'//ul[@role="listbox"]/li[@role="presentation"]')))
print('here are the instant search results:')
for r in results:
    print(r.text)
t.sleep(1)
print('and now we perform the search, by sending Enter key')
search_field.send_keys(Keys.ENTER) 

while True:
    
    results_after_pressing_enter = WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH,'//a[@class="compact-media-item-metadata-content"]')))
    print('and here are the results of the search:')
    for res in results_after_pressing_enter:
        print(res.text)
        print('___________________')
    browser.execute_script("window.scrollTo(0,document.body.scrollHeight);")
t.sleep(2)

This will print in terminal:

{'x': 486, 'y': 405}
rejected cookies
here are the instant search results:
programming is hard
programming is hard and i feel unmotivated
programming is hard to learn
programming is too hard
programming is not hard
game programming is hard
programming hardware
programming hardware with python
programming hard music
programming is easy or hard
and now we perform the search, by sending Enter key
and here are the results of the search:
Why is coding so hard...
TechLead
253K views3 years ago
___________________
coding is hard...
Joma Tech
717K views3 years ago
___________________
Why Is Programming Difficult?
Andy Sterkowitz
101K views3 years ago
[...]

Answer 2

To send a character sequence within the search bar of YouTube homepage and perform the click operation on the "search" button you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategies:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver.execute("get", {'url': 'https://www.youtube.com/'})
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#search[name='search_query']"))).send_keys("WebDriver")
driver.find_element(By.CSS_SELECTOR, "button#search-icon-legacy[aria-label='Search']>yt-icon").click()

Browser Snapshot: