0

So I have this project: It is a website with multiple WebElements on it. I find those WebElements by their class name (they all have the same one obviously) and then iterate through them to click on each of them. After clicking on them I have to click on another button "next". Some of them then open a website in a new tab (others don't). I then immediately close the newly opened tab and try to iterate through the next element when I get the StaleElementReferenceException.

Don't get me wrong here, I know what a StaleElementReferenceException is, I just don't know why it occurs. The DOM of the initial website doesn't seem to change and more importantly: The WebElement I'm trying to reach in the next iteration is still known so I can print it out, but not click on it.

I have tried working around this issue by creating a new class CustomElement to permanently "save" the found WebElements to be able to reach them after the DOM has changed but that also doesn't seem to be working.

Whatever here's some code for you guys:

def myMethod():
    driver.get("https://initialwebsite.com")
    time.sleep(1)

    scrollToBottom() #custom Method to scroll to the bottom of the website to make sure I find all webelemnts

    ways = driver.find_elements_by_class_name("sc-dYzWWc")

    waysCounter = 1

    for way in ways:
        # print("clicking: " + str(way)) ##this will get executed even if there was a new tab opened in the previous iteration....
        driver.execute_script("arguments[0].click();", way) 
        # print("clicked: " + str(way))  ##...but this won't get executed

        try:
            text = driver.find_element_by_xpath("/html/body/div[1]/div/div[2]/div/div[1]/div/div[7]/div[2]/div[" + str(entryWaysCounter) + "]/div[1]/div/div/div[1]").text
        except:
            waysCounter += 1
            text = driver.find_element_by_xpath("/html/body/div[1]/div/div[2]/div/div[1]/div/div[7]/div[2]/div[" + str(entryWaysCounter) + "]/div[1]/div/div/div[1]").text

        methode = None


        #big bunch of if and else statements to give methode a specific number based on what text reads


        print(methode)


        weiterButton = driver.find_element_by_xpath(
            "/html/body/div[1]/div/div[2]/div/div[1]/div/div[7]/div[2]/div[" + str(
                entryWaysCounter) + "]/div[2]/div/div/div/div/div/div[2]/button[2]")

        try:
            driver.execute_script("arguments[0].click();", weiterButton)
        except:
            pass

        if (methode == 19):
            time.sleep(0.2)
            try:
                driver.switch_to.window(driver.window_handles[1])
                driver.close()
                time.sleep(0.5)
                driver.switch_to.window(driver.window_handles[0])
                time.sleep(0.5)
            except:
                pass

        waysCounter += 1
        time.sleep(0.5)

And for those who are curious here's the workaround class I set up:

class CustomElement:
    def __init__(self, text, id, location):
        self.text = text
        self.id = id
        self.location = location

    def __str__(self):
        return str(str(self.text) + " \t" + str(self.id) + " \t" + str(self.location))


def storeWebElements(seleniumElements):
    result = []
    for elem in seleniumElements:
        result.append(CustomElement(elem.text, elem.id, elem.location))
    return result

I tried then working with the id and "re-finding" the WebElement ("way") by id but apparently the id that gets saved is a completely different id.

So what can I say I really tried my best, searched nearly every forum but didn't come up with a good soluation, I really hope you got one for me :)

Thanks!

MrRuck
  • 1

1 Answers1

0

Are you crawling links? If so then you want to save the destination, not the element.

Otherwise you could force the link to open in a new window (perhaps like https://stackoverflow.com/a/19152396/1387701), switch to that wind, parse the page, close the page and still have the original window open.

DMart
  • 2,401
  • 1
  • 14
  • 19
  • Yes kind of but the links aren't really attached to anything so I couldn't grab them. But they all had a similar XPath, only one number changed (counting up). So I just looped through them and searched for each Element in every loop with a different XPath. – MrRuck Oct 28 '20 at 19:30