How to generate the search results in fbref.com sending text to the search field using Python Selenium and send_keys method

Question

I am unable to retrieve any search results in fbref.com when using either of send_keys and execute_script in selenium for python using chrome web driver

This is the code ive used so far:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service 
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
import csv
from webdriver_manager.chrome import ChromeDriverManager  
from selenium.webdriver.common.action_chains import ActionChains
s=Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=s)
driver.get("https://fbref.com/en/")
element = driver.find_element(by=By.CLASS_NAME, value="ac-hint")
action = ActionChains(driver)
element1= driver.find_element(by=By.CSS_SELECTOR, value=("input[type='search']"))
action.click(on_element=element1)
action.perform() 
#element.send_keys("lionel messi")
#driver.execute_script("arguments[0].value='lionel messi'",element)
element2=driver.find_element(by=By.CSS_SELECTOR, value=("input[type='submit']"))
action.click(on_element=element2)
action.perform()```

The code is able to interact with the search button and the text is typed and the search button is clicked without any trouble but the search result is as follows:

which basically means that the search was invalid ,ive tried to search manually in the browser window opened by the driver and that gives me a successful result

Answer 1

You are doing your player name input in the wrong field, if you look closely at the html, there are 2 input fields for the search. instead of the "ac-hint", use "ac-input":

element = driver.find_element(by=By.CLASS_NAME, value="ac-input")

Answer 2

The locator strategy you have used to identify the search field doesn't identifies the desired element uniquely within the HTML DOM

---

Solution

To send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following solution:

• Code Block:

  driver.get("https://fbref.com/en/")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[type='search'][placeholder='Enter Person, Team, Section, etc']"))).send_keys("lionel messi" + Keys.RETURN)
  

• Note: You have to add the following imports :

  from selenium.webdriver.common.keys import Keys
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

• Browser Snapshot: