0

I'm trying to create a script to pull and input some information on a secure webpage, but it looks like I'm unable to find any elements on the page whatsoever. Each find_element() call would return NoSuchElementException or TimeoutError (meaning the timer on WebDriverWait expired trying to find the element).

Initially I had assumed that this was because I wasn't on the correct iframe, but my code can't find any of those either! After inspecting the page on Chrome, I was able to find one parent iframe, and then a nested iframe that I don't think is relevant.

This parent iframe is as such:

<iframe title="Main Page" id="main" name="main" src="super_long_url" slot="core-ui" style="visibility: visible;"> **Page Content** </iframe>

I've tried finding this iframe multiple ways, here are some (all seperate):

WebDriverWait(driver, 60).until(EC.frame_to_be_available_and_switch_to_it(By.ID, "main"))

time.sleep(30)
driver.switch_to.frame(By.ID, "main")

WebDriverWait(driver, 60).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[@title='Main Page']")))

frames = driver.find_element(By.TAG_NAME, 'iframe')

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(0))

To summarize, I've tried locating it by ID, XPATH, and even index. Every single attempt has either returned a TimeoutError from WebDriverWait (because it never found it) or NoSuchElementException.

I know for a fact that this "Main" iframe is the parent of all other iframes, but supposed it wasn't, shouldn't frames = driver.find_element(By.TAG_NAME, 'iframe') still return a list of elements (or at least one)?

To be clear, I'm not sure if this is an issue with exclusively iframes. I think this might be an issue with Selenium not being able to find any elements at all, including iframes.

EDIT: Weeks later, I've found the issue. Turns out the entirety of the page's elements were in a Shadow DOM tree. I had to cd (for lack of a better word) through multiple nested shadow roots until I could finally locate the iframe and switch to it. Here's how it looks in code form.

# First I located the parent div of the entire page
entryPage = driver.find_element(By.CSS_SELECTOR, "css_selector_name_123")

# Then I went through through nested shadow roots (shroots)
shroot = entryPage.shadow_root
tempDiv = shroot.find_element(By.CSS_SELECTOR, "css_selector_name_456")
shroot2 = tempDiv.shadow_root

# Then I was in the same html directory as the iframe, so I located and switched to it
iframe = shroot2.find_element(By.ID, "main")
driver.switch_to.frame(iframe)

# And from here on out, I was able to access all elements on the page just as normal
PlainMilk
  • 1
  • 1

1 Answers1

0

To locate and switch_to.frame() ideally you need to induce WebDriverWait for the frame to be available and switch to it and you can use either of the following locator strategies:

  • Using ID:

    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.ID, "main")))

  • Using NAME:

    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.NAME, "main")))

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe#main[title='Main Page']")))

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "iframe[@id='main' and @title='Main Page']")))

Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Thanks for the quick reply. Unless I'm misunderstanding, I've already done exactly that. You can see that the WebDriverWait line using ID you posted is nearly identical to the first line in my examples that I've tried. Nonetheless, I tried both examples and got a TimeoutException again. – PlainMilk Jul 25 '22 at 20:25
  • I see you've updated your comment. Unfortunately none of these work on my end. It never finds the iframe and returns a TimeoutException. I very clearly see the iframe when inspecting the webpage though, there's no mistaking it. Also, I already have all three of those exact imports. – PlainMilk Jul 25 '22 at 20:30