How to properly use loop in selenium using chrome webdriver in Python?

Question

I have a site that has multiple input fields containing URLs with the same value (X) that I want to replace with (Y). Basically, I want to automate replacing URL X with URL Y for all occurrences of URL X. There is an edit and save button to the right of each of these input fields. And each time you click save an alert pops up for each field you update. This is in short, the entire workflow I need to automate.

I'm at the point in my code that I can do it successfully for one field, but when I use a for loop it just updates one element/field twice and not the rest. FYI - I limited my for loop to two iterations using itertools.islice intentionally since I want to limit my testing to a small subset before I do a full run.

Below is a screenshot of what the page looks like: HTML

New error:

C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py:21: DeprecationWarning: use options instead of chrome_options   chrome_browser = webdriver.Chrome(service=service, chrome_options=chrome_options)

DevTools listening on ws://127.0.0.1:61787/devtools/browser/155908c1-8be2-40a1-85bf-27c44bf25506 Traceback (most recent call last):   File "C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py", line 90, in <module>
    auto_ftp_url()   File "C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py", line 57, in auto_ftp_url
    number = elem.get_attribute("id").split("_")[-1]
             ^^^^^^^^^^^^^^^^^^^^^^^^   File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webelement.py", line 177, in get_attribute
    attribute_value = self.parent.execute_script(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webdriver.py", line 500, in execute_script
    return self.execute(command, {"script": script, "args": converted_args})["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webdriver.py", line 440, in execute
    self.error_handler.check_response(response)   File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document  (Session info: chrome=109.0.5414.120) Stacktrace: Backtrace:
        (No symbol) [0x00506643]
        (No symbol) [0x0049BE21]
        (No symbol) [0x0039DA9D]
        (No symbol) [0x003A09E4]
        (No symbol) [0x003A08AD]
        (No symbol) [0x003A12E5]
        (No symbol) [0x004079F7]
        (No symbol) [0x003EFD7C]
        (No symbol) [0x00406B09]
        (No symbol) [0x003EFB76]
        (No symbol) [0x003C49C1]
        (No symbol) [0x003C5E5D]
        GetHandleVerifier [0x0077A142+2497106]
        GetHandleVerifier [0x007A85D3+2686691]
        GetHandleVerifier [0x007ABB9C+2700460]
        GetHandleVerifier [0x005B3B10+635936]
        (No symbol) [0x004A4A1F]
        (No symbol) [0x004AA418]
        (No symbol) [0x004AA505]
        (No symbol) [0x004B508B]
        BaseThreadInitThunk [0x771300F9+25]
        RtlGetAppContainerNamedObjectPath [0x77847BBE+286]
        RtlGetAppContainerNamedObjectPath [0x77847B8E+238]

PS C:\Users\me\OneDrive\Documents\Python\Web page entry automation> [15148:25476:0201/123239.502:ERROR:device_event_log_impl.cc(215)] [12:32:39.502] USB: usb_service_win.cc:415 Could not read device interface GUIDs: The system cannot find the file specified. (0x2) [15148:25476:0201/123239.557:ERROR:device_event_log_impl.cc(215)] [12:32:39.557] USB: usb_device_handle_win.cc:1046 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F) [15148:25476:0201/123239.575:ERROR:device_event_log_impl.cc(215)] [12:32:39.575] USB: usb_device_handle_win.cc:1046 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

Below is my code:

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.alert import Alert
import itertools


def auto_ftp_url():
    username, password = login()

    chrome_options = Options()
    chrome_options.add_experimental_option("detach", True)

    service = Service(
        r"C:\Users\me\OneDrive\Documents\chromedriver_win32\chromedriver.exe"
    )

    chrome_browser = webdriver.Chrome(service=service, chrome_options=chrome_options)
    chrome_browser.get("https://somelink")

    # send text to username field
    userElem = chrome_browser.find_element(by="id", value="Username")
    userElem.click()
    userElem.send_keys(f"{username}")

    # send password to password field
    passwordElem = chrome_browser.find_element(by="id", value="Password")
    passwordElem.click()
    passwordElem.send_keys(f"{password}")

    # click login
    loginElem = chrome_browser.find_element(by="id", value="Button1")
    loginElem.click()

    # add some delay (2 seconds) to the webdriver so that the elements will load
    chrome_browser.implicitly_wait(2)

    # click on Admin button on left navigation pane
    adminElem = chrome_browser.find_element(By.XPATH, "//*[@id='btnAdminPage']")
    adminElem.click()

    # click on Data FTP button in the top center
    data_ftp_Elem = chrome_browser.find_element(
        By.XPATH, "//*[@id='menu-outer']/div[12]/a/div"
    )
    data_ftp_Elem.click()

    # find FTP URL with a value of 'aftersoftna.com'
    ftp_url_Elem = chrome_browser.find_elements(
        By.XPATH, "//*[@value='aftersoftna.com']"
    )
    for elem in itertools.islice(ftp_url_Elem, 2):
        if elem:  # if the value is 'aftersoftna.com' then do the following
            # click edit button
            editElem = chrome_browser.find_element(
                By.XPATH, "//*[@id='ContentPlaceHolder1_gvFTP_btnSave_7']"
            )
            editElem.click()
            # use WebDriverWait to wait for the element to be clickable
            textElem = WebDriverWait(chrome_browser, 20).until(
                EC.element_to_be_clickable(
                    (By.XPATH, "//*[@id='ContentPlaceHolder1_gvFTP_txtFtpUrl_7']")
                )
            )
            textElem.click()
            textElem.clear()
            textElem.send_keys(
                "catalog.com"
            )  # after clearing the old URL, enter the new URL
            # click save
            saveElem = chrome_browser.find_element(
                By.XPATH, "//*[@id='ContentPlaceHolder1_gvFTP_btnSave_7']"
            )
            saveElem.click()
            # popup or alert message comes up confirming changes saved. click ok
            WebDriverWait(chrome_browser, 10).until(EC.alert_is_present())
            chrome_browser.switch_to.alert.accept()


def login():
    username = input("Username: ")
    password = input("Password: ")
    return username, password


auto_ftp_url()

Answer 1

Without seeing the page it's impossible to know but I'm going to guess that it's because you are using the same locators for editElem, textElem, and saveElem for each URL. Each of them end in "_7" but I would assume that there are multiple elements on the page, one set of three for each URL from what you are describing, e.g. "_8", "_9", etc. If this is true then your locators need to be relative to the URL that you want to edit.

If I were to guess even further, you should be able to get your list of ftp_url_Elem elements, grab the ID from each in the loop, and then extract the final number, e.g. 7 from ContentPlaceHolder1_gvFTP_txtFtpUrl_7, and then extrapolate that to the other ID, ContentPlaceHolder1_gvFTP_btnSave_7.

To get the magic number that corresponds to the row number, we can

1. Grab the first ID using elem.get_attribute("id"), ContentPlaceHolder1_gvFTP_txtFtpUrl_7
2. Split that ID by "_" resulting in ['ContentPlaceHolder1','gvFTP','txtFtpUrl','7']
3. Grab the last instance from that list which is the desired row number, '7'
4. Now we can append this number to the "prefix" of the ID for the other elements to get the correct element on the matching row of the table, e.g. ID prefix for the Edit/Save button, ContentPlaceHolder1_gvFTP_btnSave_, and add '7' to get ContentPlaceHolder1_gvFTP_btnSave_7

Then you just repeat that last step for all the elements on that row that you need to click.

Here's all the code as an example

for elem in itertools.islice(ftp_url_Elem, 2):
    if elem:  # if the value is 'aftersoftna.com' then do the following
        # get row number from element ID
        row_number = elem.get_attribute("id").split("_")[-1]
        # click edit button
        editElem = chrome_browser.find_element(
            By.XPATH, "//*[@id='ContentPlaceHolder1_gvFTP_btnSave_" + row_number + "']"
        )
        ... and so on

If this is enough to point you in the right direction to fix it yourself, great. If not, please post a few example rows of HTML that contain the edit, URL field, and save button so we can create code and relative locators.