3

I am trying to capture some website elements using selenium in python, and I am using try/except in case there is a specific element that cannot be found in that particular page. This all well and good, however the script still throws a NoSuchElementException even though I am expecting it and have told the script to handle it a certain way or pass.

The only thing I can think that may be an issue is that this try/except is nested inside another try/except, so that i goes like this

for month in (month_start, month_end):

    for date in (date_start, date_end):

        try: #this is just a general try in case there is a breakdown in finding one of the elements
            driver.get(url)
            results = driver.find_elements_by_xpath("""//*[@class="results"]/div""")

            for result in results:
                sample_element = result.find_element_by_xpath("blah").text

                #seems to be where the problem starts
                specific_element = ""
                try:
                    #this is a specific element that I know may not exist
                    specific_element = result.find_element_by_xpath(""".//*[@class="specific"]/div""").text
                except NoSuchElementException:
                    specific_element = ""
                    #I have tried pass instead as well with no luck
                #throws an error here and won't continue if the specific element is not found
        except:
            #pretty much a copy of the above try with some changes

So I generally think that I have a decent understanding of the try/except function in python but this is doing my head in. The "result in results" loop will happily continue until it doesn't find specific_element and just throws a "selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:"

If being nested inside the try/except is the cause of the whole issue, could you please explain why that is the case, and recommend a possible solution that I could research or implement? Or maybe I have missed something fundamental.

Christian Dean
  • 22,138
  • 7
  • 54
  • 87
Ryan N.
  • 356
  • 4
  • 17
  • Do you import NoSuchElementException? – Niyamat Ullah Jan 10 '18 at 01:31
  • Maybe you are using wrong `NoSuchElementException`. Maybe there is another `NoSuchElementException` with the same name as what you get. Could you try explicitly importing `from selenium.common.exceptions import NoSuchElementException`? – Sraw Jan 10 '18 at 01:36
  • @NiyamatUllah Yes I have. It doesn't even seem to recognise the try/except, and won't continue to the except after the try has failed. I have tried just "except:" as well and still throwing the same error. – Ryan N. Jan 10 '18 at 01:38
  • Do you know what the purpose of the triple double quotes is? – Rescis Jan 10 '18 at 02:21
  • @Rescis from my understanding the the triple double quote is the same as a single double quote but it can contain line breaks. For me personally it's just an easy way of highlighting strings inside of long lines. – Ryan N. Jan 10 '18 at 03:08
  • Alright. The nested exception handling shouldn't be an issue from what I understand: https://stackoverflow.com/questions/17015230/are-nested-try-except-blocks-in-python-a-good-programming-practice. For the sake of making sure though, I'd try wrapping the interior exception bit in a method and calling it. – Rescis Jan 10 '18 at 03:17
  • could you update the code with import statement for `NoSuchElementException` ? – Gaurang Shah Jan 10 '18 at 15:35

2 Answers2

1

I don't do python but if it were me, I would remove all the try/catches and replace them with find_elements_* and check for empty list. For example

replace

specific_element = result.find_element_by_xpath(""".//*[@class="specific"]/div""").text

with

elements = result.find_elements_by_xpath(".//*[@class='specific']/div")
if elements
    specific_element = elements[0]

This basically just makes sure that any element is found before accessing it and you can avoid all the try/catches. I'm of the opinion that exceptions should be exceptional... rare. They shouldn't be expected or used as flow control, etc.

JeffC
  • 22,180
  • 5
  • 32
  • 55
  • There is no need to add the text **I don't do python but if it were me** as an introduction to any of your answers. On StackOverflow it is nothing but a **noise** where we are looking for good Answers. – undetected Selenium Jan 12 '18 at 12:45
0

You have to take care of a couple of points here :

  • There is no absolute necessity to try driver.get(url)

  • The issue is in the following block :

    try:
    specific_element = result.find_element_by_xpath(""".//*[@class="specific"]/div""").text

Analysis

You have tried a chain of events before the try() actually comes into action :

  • Evaluate 

    result.find_element_by_xpath(""".//*[@class="specific"]/div""")

  • Next 

    result.find_element_by_xpath(""".//*[@class="specific"]/div""").text

  • Next 

    specific_element = result.find_element_by_xpath(""".//*[@class="specific"]/div""").text

So,

  • Once result.find_element_by_xpath(""".//*[@class="specific"]/div""") fails though NoSuchElementException is raised, try still remains silent.

Solution

A simple solution would be :

try:
    specific_element = result.find_element_by_xpath(""".//*[@class="specific"]/div""")
    specific_element_text = specific_element.text
    # other works

except NoSuchElementException:
    specific_element = ""
    # other works
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352