Webdriver Wait or equivalent - with attributes

Question

I have a snippet of code that loops through links in a page using Selenium. I have replaced Time.sleep(3) with a WebDriverWait() to try to speed it up (as I'm running through 50 pages) but I can't get it to work. For some reason implicitly_wait(30) doesn't do the trick.

Without any Time.sleep() or WebDriverWait() the code generates a stale element reference error when trying to declare the variable "elems", as the driver can't find any elements by xpath - not surprising.

The WebDriverWait() that I've used here solves that problem, but then the code runs into another: On the last line of the nested for-loop, it throws another stale element reference error. It seems that although the elements are loaded when WebDriverWait() lets the code proceed, the attributes of these elements are not necessarily ready.

My objective is to make the code as fast as possible, so Time.sleep(10) or something similar wouldn't be very appealing. How do I smoothly make sure that all elements and their attributes are loaded before continuing? And why could it be that implicitly_wait(30) does not solve this issue?

EDIT: The URLs in question: From http://www.oddsportal.com/baseball/usa/mlb-2017/results/#/page/1/ to http://www.oddsportal.com/baseball/usa/mlb-2017/results/#/page/50/

PageRange = range(1, 50)
for page in PageRange:
    URL = "http://www.oddsportal.com/baseball/usa/mlb-" + season + "/results/#/page/" + str(page) + "/"
    driver.get(URL)
    tryElement = EC.presence_of_element_located((By.XPATH, "//a[@href]"))
    WebDriverWait(driver, 30).until(tryElement)
    elems = driver.find_elements_by_xpath("//a[@href]")
    for elem in elems:
        #This line I can't get not to throw a stale element reference 
        link = elem.get_attribute("href")

you can try "EC.title_contains" to make sure to page loaded properly. — Narendra, Feb 12 '18 at 12:39
I don't think this question should be marked as a duplicate. The listed duplicate question does not have a marked answer and the above question is unique enough to be it's own. @DanielSlätt, if you are still dealing with this issue please let me know. — PixelEinstein, Feb 15 '18 at 01:58
Hi! Yes, I have not yet been able to solve this. I wonder if there's any additional information I should give? — Daniel Slätt, Feb 15 '18 at 12:08

score 0 · Answer 1 · answered Feb 12 '18 at 13:38

0

As per the reference code block you have provided you can use the following code block to avoid StaleElementReferenceException :

PageRange = range(1, 50)
for page in PageRange:
    URL = "http://www.oddsportal.com/baseball/usa/mlb-" + season + "/results/#/page/" + str(page) + "/"
    driver.get(URL)
    elems = WebDriverWait(driver, 30).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@href]")))
    for elem in elems:
        print(elem.get_attribute("href"))

answered Feb 12 '18 at 13:38

undetected Selenium

183,867
41
278
352

Hi! Thanks for the suggestion! For some reason, the code gets stuck here until timeout. I tried changing visibility to presence and it throws the stale element error as usual. – Daniel Slätt Feb 12 '18 at 13:56
Well, this outcome was written on the wall in the absence of the relevant HTML. If you can update the question with the relevant HTML possibly SO volunteers can construct an working answer for you. – undetected Selenium Feb 12 '18 at 13:58
Of course. I have added the URL's in question to the original post. – Daniel Slätt Feb 12 '18 at 14:05

Webdriver Wait or equivalent - with attributes

1 Answers1