selenium, python - WebDriverWait times out 60% of the time no matter what I set the delay to

Question

Traceback

09:32:01 Traceback (most recent call last):
  File "/home/mike/projects/work/blacklight/venv/lib/python3.8/site-packages/rq/worker.py", line 975, in perform_job
    rv = job.perform()
  File "/home/mike/projects/work/blacklight/venv/lib/python3.8/site-packages/rq/job.py", line 696, in perform
    self._result = self._execute()
  File "/home/mike/projects/work/blacklight/venv/lib/python3.8/site-packages/rq/job.py", line 719, in _execute
    return self.func(*self.args, **self.kwargs)
  File "./sba_scraper/scrape.py", line 69, in scrape
    any_minority_owned_checkbox = WebDriverWait(driver, delay).until(
  File "/home/mike/projects/work/blacklight/venv/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

Code

driver = webdriver.Firefox(firefox_profile=profile, options=options)
url = 'https://web.sba.gov/pro-net/search/dsp_dsbs.cfm'
driver.get(url)
delay = 60
checkbox = WebDriverWait(driver, delay).until(
    EC.presence_of_element_located((By.ID, 'EltCbtMin')))

Whether I set the delay to 3 or 10 or 60, Most of the time I get the exception above, that it timed out before it could find the element. Obviously it works 40% of the time though. Can anyone tell me what is wrong?

score 0 · Accepted Answer · answered Feb 15 '21 at 09:59

To click on the checkbox associated with text as Any Minority Owned you can use either of the following Locator Strategies:

Using css_selector:

driver.get('https://web.sba.gov/pro-net/search/dsp_dsbs.cfm')
driver.find_element(By.CSS_SELECTOR, "input#EltCbtMin[aria-labelledby='LabCbtMin']").click()

Using xpath:

driver.get('https://web.sba.gov/pro-net/search/dsp_dsbs.cfm')
driver.find_element(By.XPATH, "//input[@id='EltCbtMin' and @aria-labelledby='LabCbtMin']").click()

Ideally, to click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get('https://web.sba.gov/pro-net/search/dsp_dsbs.cfm')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#EltCbtMin[aria-labelledby='LabCbtMin']"))).click()

Using XPATH:

driver.get('https://web.sba.gov/pro-net/search/dsp_dsbs.cfm')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@id='EltCbtMin' and @aria-labelledby='LabCbtMin']"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Browser Snapshot:

selenium, python - WebDriverWait times out 60% of the time no matter what I set the delay to

1 Answers1