Selenium + Python: Unable to catch a TimeoutException with Try & WebDriverWait

Question

Python version: 3.10; Selenium Webdriver: Firefox; IDE: PyCharm 2021.3.2 (CE); OS: Fedora 35 VM

I am writing a python selenium script to scrape data from a website. I would like to navigate through a website, find an element & print it. I am able to do this when the element is present. My problem is that sometimes the element is not present & I get an exception. If the element is not present when I use this code:

 RemainDeductible = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

the script produces the following exception:

 Traceback (most recent call last):
  
 File "...PythonSeleniumScript.py", line 152, in <module>
    RemainDue = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

 File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 89, in until
    raise TimeoutException(message, screen, stacktrace)

 selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.find/</<@chrome://remote/content/marionette/element.js:300:16

If the Timeout Exception occurs I would like to catch the exception & set the "Remain Due" variable to the string "BLANK". I still get the following Timeout Exception if I use Try in my code as follows:

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except TimeoutException:
    RemainDue = "BLANK"

I still get the Timeout Exception. I thought my code would catch the exception. Why is it not catching it???

 Traceback (most recent call last):
  
 File "...PythonSeleniumScript.py", line 155, in <module>
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 89, in until

    raise TimeoutException(message, screen, stacktrace)

 selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.find/</<@chrome://remote/content/marionette/element.js:300:16

The relevant HTML for the element I am trying to locate is:

<input id="b8-b36-Input_RemainAmtYr1"
class="form-control OSFillParent" data-
input="" disabled="" type="text"
style="margin-top: 5px;" value="$10.50">
event

Of course the code does work when the element is present. My issue is that the code crashes with the Timeout Exception when the element is not present. When the Timeout Exception occurs how do I catch the Timeout Exception & set the "Remain Due" variable to the string "BLANK"?

score 3 · Accepted Answer · answered Mar 21 '22 at 12:24

3

I would suggest several things here:

instead of

except TimeoutException:

Try using

except:

I understand that it's preferable to catch a specific exception type rather any exception, but I will still advice doing that here.
2) I would suggest using visibility_of_element_located instead of presence_of_element_located since presence_of_element_located will wait for presence of such element on the page only while it may still not be completely rendered. While visibility_of_element_located will wait for much more mature element state, when it is visible.
So instead of

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except TimeoutException:
    RemainDue = "BLANK"

I would suggest you using

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except:
    RemainDue = "BLANK"

Also make sure the id value of b8-b36-Input_RemainAmtYr1 is fixed, not changing.

answered Mar 21 '22 at 12:24

Prophet

32,350
22
54
79

The id value of b8-b36-Input_RemainAmtYr1 is fixed as long as the element is present. When the element is not present of course there is no id either. – Auditor Mar 21 '22 at 13:49
Are you sure you keep a correct indentation there? If yes (and you are 100% sure) - can you share more your code? Because this code itself should work.... – Prophet Mar 21 '22 at 13:53
So, what was the issue? – Prophet Mar 21 '22 at 14:03
Changing from ``except TimeoutException:`` to ``except :`` did work. Much appreciated but why was this necessary? What other exception could it have been generating? How do I print the exceptions without having it stop the code? – Auditor Mar 21 '22 at 14:05
The first time I changed to ``except:`` it did not work because I forgot to comment out the original code ``RemainDeductible = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))``. Sorry for the confusion. – Auditor Mar 21 '22 at 14:07
1

here https://stackoverflow.com/questions/4690600/python-exception-message-capturing you can see how to print the exception message. there are more questions / tutorials about that – Prophet Mar 21 '22 at 14:11
1

In order to see why the `except TimeoutException:` did not work I would need to debug your code, but you didn't give all the details and all your code here. But you can print the actual error using the `except Exception as e: # work on python 3.x logger.error('Failed to upload to ftp: '+ str(e))` or similar as described there – Prophet Mar 21 '22 at 14:14

Selenium + Python: Unable to catch a TimeoutException with Try & WebDriverWait

1 Answers1