Selenium .click() works very inconsistent

Question

Website: https://www.annualreports.com/Companies?

I want to identify the links on the website using the linktext, e.g. a companies name and click the respective link:

company = "1-800-FLOWERS"
browser.get(r'https://www.annualreports.com/Companies?')
continue_link = browser.find_elements(By.XPATH,f"//span[@class='companyName'][.//a[contains(.,'{company}')]]")
if len(continue_link) == 0:
    print("No Firm Found")
elif len(continue_link) > 1:
    print("Multiple Firms Found")
elif len(continue_link) == 1:
    print("Firm Found")
    continue_link[0].click()

This works just fine in some cases, for example for the company in the above code sample, "1-800-FLOWERS".
For other companies, the button won't be clicked, even though an element is found. Replacing "1-800-FLOWERS" with the company "3D Systems" leads to a case where the link is not clicked. I haven't identified a potential pattern with respect to the companies that might explain this behavior.
I also tried all kinds of possible solutions from this thread without any success.

Any help is much appreciated

score 1 · Accepted Answer · answered Dec 09 '22 at 12:17

You need to click not on span but on a element inside it containing the link.
Also, it would be better to first scroll the element into the view and only then to click on it.
This worked for me:

import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 30)

company = "3D Systems"
url = "https://www.annualreports.com/Companies?"
driver.get(url)
#
continue_link = wait.until(EC.presence_of_all_elements_located((By.XPATH, f"//span[@class='companyName'][.//a[contains(.,'{company}')]]/a")))
if len(continue_link) == 0:
    print("No Firm Found")
elif len(continue_link) > 1:
    print("Multiple Firms Found")
elif len(continue_link) == 1:
    print("Firm Found")
    driver.execute_script("arguments[0].scrollIntoView();", continue_link[0])
    time.sleep(0.4)
    continue_link[0].click()

The result is

You never fail to amaze me with your helpful and easy-to-follow answers. Thank you very much :) — xxgaryxx, Dec 09 '22 at 12:43

Selenium .click() works very inconsistent

1 Answers1