Selenium Timeout Exception not working as intended

Question

I'm new to Python, so not sure what is wrong here. From my understanding, Selenium fails to locate an element sometimes when the window is not focused (I switched to headless due to that). So I am trying to work around that and make it work 100% of the time.

This is what the error looks like:

selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
Backtrace:
        Ordinal0 [0x00986903+2517251]
        Ordinal0 [0x0091F8E1+2095329]
        Ordinal0 [0x00822848+1058888]
        Ordinal0 [0x0084D448+1233992]
        Ordinal0 [0x0084D63B+1234491]
        Ordinal0 [0x00877812+1406994]
        Ordinal0 [0x0086650A+1336586]
        Ordinal0 [0x00875BBF+1399743]
        Ordinal0 [0x0086639B+1336219]
        Ordinal0 [0x008427A7+1189799]
        Ordinal0 [0x00843609+1193481]
        GetHandleVerifier [0x00B15904+1577972]
        GetHandleVerifier [0x00BC0B97+2279047]
        GetHandleVerifier [0x00A16D09+534521]
        GetHandleVerifier [0x00A15DB9+530601]
        Ordinal0 [0x00924FF9+2117625]
        Ordinal0 [0x009298A8+2136232]
        Ordinal0 [0x009299E2+2136546]
        Ordinal0 [0x00933541+2176321]
        BaseThreadInitThunk [0x770EFA29+25]
        RtlGetAppContainerNamedObjectPath [0x77B47A9E+286]
        RtlGetAppContainerNamedObjectPath [0x77B47A6E+238]

So I tried to find a solution to this, which is to rerun/attempt again, but instead, it goes really haywire:

2021-12-22 04:50:20.920372 Signed in
Password input crashed. Retrying now...
2021-12-22 04:50:31.234082 Loaded profile page
Username input crashed. Retrying now...
2021-12-22 04:52:12.192628 Signed in
Password input crashed. Retrying now...
2021-12-22 04:52:22.473065 Loaded profile page
2021-12-22 04:52:23.174232 Signed in
Password input crashed. Retrying now...
2021-12-22 04:52:33.452513 Loaded profile page

Here's the code:

import tkinter as tk

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.chrome.options import Options

from selenium.webdriver.chrome.service import Service

from selenium.common.exceptions import TimeoutException

import threading

import time

from time import sleep

import datetime

options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument("start-maximized")

root = tk.Tk()

app_width = 300
app_height = 320

screen_width = root.winfo_screenwidth()
screen_height = root.winfo_screenheight()

x = (screen_width / 2) - (app_width / 2)
y = (screen_height / 2) - (app_height / 2)

root.geometry(f'{app_width}x{app_height}+{int(x)}+{int(y)}')

#
ser = Service("C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=ser, options=options)
wait = WebDriverWait(driver, 50)

testbtn_txt = tk.StringVar()
testbtn = tk.Button(root, textvariable=testbtn_txt, command=lambda:open_browser_func(), font="Arial", bg="#808080", fg="white", height=1, width=10)
testbtn_txt.set("Test")
testbtn.grid(row=10, column=0, columnspan=2, pady=5, padx=5)


def open_browser_func():
    global driver

    driver.get("https://twitter.com/i/flow/login")
    sleep(5)

    try:
        loginuser  = wait.until(EC.visibility_of_element_located((By.NAME, "text")))
        loginuser.send_keys("Username", Keys.RETURN)
    except TimeoutException:
        def retry_sign_in():
            print("Username input crashed. Retrying now...")
            driver.quit
            return open_browser_func()
        retrytimer = threading.Timer(5, retry_sign_in)
        retrytimer.start()

    try:
        loginPassword = wait.until(EC.visibility_of_element_located((By.NAME, "password")))
        loginPassword.send_keys("Password", Keys.RETURN)
    except TimeoutException:
        def retry_sign_in2():
            print("Password input crashed. Retrying now...")
            driver.quit
            return open_browser_func()
        retrytimer2 = threading.Timer(5, retry_sign_in2)
        retrytimer2.start()

    ct = datetime.datetime.now()
    print(ct, "Signed in")

    def logged_in_shot():
        driver.get_screenshot_as_file("logged_in_shot.png")
        driver.get("https://twitter.com/Username")
        
        ct2 = datetime.datetime.now()
        print(ct2, "Loaded profile page")

    logged_in_timer = threading.Timer(10, logged_in_shot)
    logged_in_timer.start()
    
    return driver

root.mainloop()

Answer 1

With the exception handling here, you may want to implement a limit to how many times it tries, or not call the original function the way you are.

try:
    loginuser  = wait.until(EC.visibility_of_element_located((By.NAME, "text")))
    loginuser.send_keys("Username", Keys.RETURN)
except TimeoutException:
    def retry_sign_in():
        print("Username input crashed. Retrying now...")
        driver.quit
        return open_browser_func()
    retrytimer = threading.Timer(5, retry_sign_in)
    retrytimer.start()

You are retrying here, which is fine, but if you are returning the open_browser_func() each time, if it continues to fail it will continue to run that function. If you pass a limit (IE, 3 times) and enforce that, you might resolve this issue.

For the original question here, I have used two solutions to help with elements not being loaded yet in Selenium-

1. Using a sleep or wait function, which works but can be sloppy.
2. Creating a function to wait until the element is in view. This tends to work best.