Selenium can't find dynamially loaded input field element even after waiting for that specific element

Question

I'm trying to access an input field of username (of login page) with Selenium. The page is JavaScript based. driver.get() wait by default to load the complete page. In my case, it is unable to load that. I can inspect the element on browser (firefox) and I get this.

<input type="text" autocomplete="username" name="username">

I tried to wait for that specific element with EC.presence_of_element_located.

Code trials:

driver = webdriver.Firefox()
driver.get(url) 
delay = 10 # seconds
try:
    myElem = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.NAME, 'username')))
    print("Page is ready!")
except TimeoutException:
    print("Loading took too much time!")

print(driver.page_source)

I get Loading took too much time!. Even though the element is there as I can inspect it in the browser. I also tried EC.presence_of_element_located((By.TAG_NAME, 'input'))) but It also can't find the tag.

Update from the comments: url='https://drive.inditex.com/drfrcomr/login'

That element is in a #shadow root. See https://stackoverflow.com/questions/73671844/cant-click-on-accept-cookies-how-to-find-correct-frame/73671977#73671977 — Barry the Platipus, Jan 13 '23 at 19:23
check this [answer](https://stackoverflow.com/questions/70558947/python-selenium-alert-like-authentication-pop-up) tell me if that works for you! — William Castrillon, Jan 13 '23 at 20:13
@BarrythePlatipus That's not a _`#shadow root`_ really but _Windows Authentication_. — undetected Selenium, Jan 13 '23 at 20:43
@undetectedSelenium you're wrong: https://imgur.com/MlWp2iT.png. That *is* a shadow root element. — Barry the Platipus, Jan 13 '23 at 22:05
@BarrythePlatipus OP isn't pointing to any `
` but an `` field. However accessing the mentioned [url](https://drive.inditex.com/drfrcomr/login) opens the [Windows Authentication](https://i.stack.imgur.com/H7Gh3.png). — undetected Selenium, Jan 13 '23 at 22:10
No @undetectedSelenium. Look closely at the following printscreen: https://imgur.com/NtdmDbQ.png. See the input highlighted, see the url. That specific element is within a shadow root. — Barry the Platipus, Jan 13 '23 at 22:13
@BarrythePlatipus Possibly the UI renders differently in APAC, UK and US :/ — undetected Selenium, Jan 13 '23 at 22:14
@everyone, Basically, when you open the website with Chrome, you get the login prompt [login prompt](https://i.ibb.co/MZ4zypb/chrome.jpg). But when you use Firefox, you don't get it and you are directly into the [login form page](https://i.ibb.co/D8yNJT2/firefox.jpg). The login prompt is of no use. The login form page is the actual login page. I can click on cancel button on login prompt to proceed further to login form page in chrome. Also, I can use `username:password@url` to get rid of login prompt in chrome. But, again login page form happens after that. — Shaida Muhammad, Jan 14 '23 at 14:21

undetected Selenium · Accepted Answer · 2023-01-14T17:39:01.110

To interact a clickable element instead of presence_of_element_located() you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Using NAME:

myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.NAME, "username")))

Using CSS_SELECTOR:

myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='username']")))

Using XPATH:

myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@name='username']")))

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

However on accessing the website I find the username field implemented through Windows Authentication

inditex

Solution

Incase of Basic Authentication you need to embed the username and password within the url as follows:

driver.get("http://username:password@drive.inditex.com/drfrcomr/login")

Update

Additionally the username field is within nested #shadow-root (open)

To send a character sequence to the username field you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:

Code Block:

driver.get("http://username:password@drive.inditex.com/drfrcomr/login")
time.sleep(10)
item = driver.execute_script('''return document.querySelector('glyph-splash.hydrated').shadowRoot.querySelector('glyph-login-form.hydrated').shadowRoot.querySelector('input[name="username"]')''')
item.send_keys("ShaidaMuhammad")

Basically, the `driver.get("http://username:password@drive.inditex.com/drfrcomr/login")` will only remove the login prompt, but after that, the actual login page happens. Even if you just click on `cancel` on the login prompt, you will get the actual login page form where the `username` and `password` fields are happened. Now the problem start from their. I can't access those field even I used `EC.presence_of_element_located`. I can inspect the element in the browser but can't access it in the code as the field is not accessable somehow. — Shaida Muhammad, Jan 14 '23 at 13:22
Also, I tried this method. I used `driver.get("http://username:password@drive.inditex.com/drfrcomr/login")` to get rid of that prompt but then I ended up on the same login form and same problem happened as I explain in the question. Also, I used `EC.element_to_be_clickable` i.e. copy pasted your code. — Shaida Muhammad, Jan 14 '23 at 13:28
It didn't work in first place i.e. I pasted the code above the code I wrote in the question. Then, I posted below the code. Interestingly, the code worked then. Even though I got the same error again `Loading took too much time!`. but the keys were successfully sent. Now the problem is after clicking login, it stuck on authentication. Is there any problem in sending keys to the fields. Or the website has some blocking mechanism? — Shaida Muhammad, Jan 14 '23 at 20:05

score 0 · Answer 2 · edited Jan 13 '23 at 20:44

0

You can do as stated here, so your code would looks like

url='https://user:password@drive.inditex.com/drfrcomr/login'
driver.get(url)

I tried with Chrome and it worked for me!

edited Jan 13 '23 at 20:44

undetected Selenium

183,867
41
278
352

answered Jan 13 '23 at 20:23

William Castrillon

52
5

It will just clearn the username/password prompt. This is not the actual login form. I can click cancel on the prompt and then get to the actual login form. Even in Firefox, you don't get that prompt. I used firefox in the code. In firefox, we get to the login page directly. – Shaida Muhammad Jan 14 '23 at 13:14

score 0 · Answer 3 · edited Jan 13 '23 at 20:44

0

If you can get around that popup you may try:

myElem = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.XPATH, "//*[.='Username']")))

edited Jan 13 '23 at 20:44

undetected Selenium

183,867
41
278
352

answered Jan 13 '23 at 20:24

Dominik

26
5

Selenium can't find dynamially loaded input field element even after waiting for that specific element

3 Answers3

Solution

Update