iframe element missing from selenium

Question

I am trying to do some automated functionality using python selenium, and I'm coming across some weird behavior.

General layout of the html:

<html>
  <body>
    <div class="parent">
      <iframe style="display: none"> ... </iframe>
      <iframe style="display: none"> ... </iframe>
      <iframe style="display: block">
        #document
        ...
        <div class="someClass"> ... </div>
      </iframe>
      <iframe style="display: none"> ... </iframe>
      <iframe style="display: none"> ... </iframe>
    </div>
  </body>

Now, each iframe actually has the same inner html, and the code from the website seems to be randomly choosing which iframe is getting the display="block". However, I can't find any of the iframes.

I tried a standard way: iframe = driver.find_elements_by_xpath("//iframe[contains(@style, 'display:block')]")

That failing, I then tried just to find any iframe: driver.find_element_by_tag_name("iframe")

Neither of those found any iframe elements. I'm seeing the following error:

Traceback (most recent call last):
  File "myfile.py", line 60, in <module>
    iframe = driver.find_element_by_xpath("//iframe[contains(@style, 'display: block')]")
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
    'value': value})['value']
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "//iframe[contains(@style, 'display: block')]" is: [object HTMLIFrameElement]. It should be an element.
  (Session info: chrome=93.0.4577.63)

Any thoughts on why the xpath is returning [object HTMLIFrameElement] and why I can't access that as I do other objects when searching by xpath?

Edit

New code option 1:

iframes = driver.find_elements_by_xpath(".//iframe[contains(@style,'display: block')]")

This still throws the exact same error as above

New code option 2:

parent = driver.find_element_by_xpath("//div[@class='parent']")
iframes = parent.find_elements_by_tag_name("iframe")
// when I print typeof iframes here, it's a list of dicts
// find the right index. Here, for simplicity, I just set it a default value
index = 4
// ...
driver.switch_to.frame(iframes[index])

I get the following error:

Traceback (most recent call last):
  File "myfile.py", line 76, in <module>
    driver.switch_to.frame(iframe)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\switch_to.py", line 89, in frame
    self._driver.execute(Command.SWITCH_TO_FRAME, {'id': frame_reference})
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: missing 'ELEMENT'
  (Session info: chrome=93.0.4577.82)

And when I print out iframes:

[{}, {}, {}, {}, {u'ontouchmove': {}, u'ontouchstart': {}}, {}, {}, {}, {}, {}]

For reference, this is the page I'm trying to hit. Sometimes you have to refresh a few times to get the challenge, and it comes far more frequent when using selenium. Also, using headless mode causes the challenge to happen everytime... https://catalog.usmint.gov/coins/coin-programs/morgan-and-peace-silver-dollar-coins/

Does this answer your question? [Switch to an iframe through Selenium and python](https://stackoverflow.com/questions/44834358/switch-to-an-iframe-through-selenium-and-python) — Striter Alfa, Sep 27 '21 at 18:20
No. I can't find the element before I even try to switch to it, as I mentioned in my question — lcta0717, Sep 28 '21 at 05:37
Have you exported the html source in selenium to be sure the elements are present for a manual inspection/search? The target web page may not be rendering as you expect via the emulated browser vs. how it is rendered in a full desktop browser. I remember chasing a similar issue extensively in the past only to discover that the webdriver browser was rendering totally different content vs a full chrome browser. — Chris Lindseth, Sep 30 '21 at 03:05
I did try to export the html, but it's being dynamically rendered. In the exported html, I do see some iframe elements, but all of them have no children. This differs from what I see when I look at the developer options, though I think if it's dynamically rendered I would expect to see a different, right? — lcta0717, Sep 30 '21 at 19:06

Luan Naufal · Answer 1 · 2021-10-04T12:42:35.427

1

The only situation I've seen like that was when trying to access an iframe from any ads of something similar, that was disappearing in the moment I was trying to get them.

I always solved that with the getElementsByTagName("iframe"), so try to wait a little more for the page to load before loading it, just to be sure the iframe was totally initialised before running it. One way to do this was already discussed on this question

Also, here the official doc on the wait patterns for python: https://selenium-python.readthedocs.io/waits.html

PS: just tested your example html page and I could easily get them on my browser when using document.getElementsByTagName("iframe"), as you can see on the image below, so most probably you're running into one of the issues I've mentioned above, since your Selenium should be able to see them, assuming they are static and don't vanish and that your page fully loaded:

Extra Details

In your case, if you're receiving as attribute the HTMLIFrameElement instead of a simple iframe tag, it means you're dealing with a Web interface, from which you can access their attributes directly, and it means that indeed you found an iframe on the page. You can use its properties to access the native APIs, and it has a .src property, reflecting the URL that's being loaded by it, and in many cases you could open this URL in a different page, and get what's rendered by it directly (unless the URL contains some CORS block). Also, there's indeed some bugs on Selenium with Chrome related to WaitForPageToLoad and this can be fixed using other methods, as described here, though I don't think it's your current issue.

edited Oct 04 '21 at 12:42

answered Sep 30 '21 at 02:32

Luan Naufal

1,346
9
15

I definitely think the website is doing something. It's a type of captcha page, so I think somehow they're tricking selenium into not being able to see what the screen renders. – lcta0717 Sep 30 '21 at 19:07
Is my output (```[object HTMLIFrameElement]```) the standard output when the element is not found, or am I somehow grabbing the object at the wrong level? – lcta0717 Sep 30 '21 at 19:08
@lcta0717, I just updated my answer to include the points related to the `HTMLIFrameElement`. Please take a look if that helps. – Luan Naufal Oct 04 '21 at 12:43
How would I access the element though? The selenium return is throwing an error because it's not returning an element – lcta0717 Oct 05 '21 at 13:11
@lcta0717 but you've mentioned that you can access an `object HTMLIFrameElement`, so you got the iFrame Element returned from web apis, as the link I've added details. The problem, possibly, is related to your `xpath` expression. Try running `driver.find_elements_by_tag_name("iframe")`, but please pay attention on the `elements`, in plural form. Because if this page has multiple `iframes`, you might be getting the reference of one that's vanished after the page loads or any related issues, that might prevent you to access it. – Luan Naufal Oct 05 '21 at 18:47
I made an edit update - I'm now no longer throwing an error, and it seems to be finding the list of iframes. but it's not returning them in the proper fashion (it's returning a list of dicts). – lcta0717 Oct 06 '21 at 08:07
In addition, searching by xpath is still throwing the same error, even with using the ```find_elements_by_xpath``` function (note the s) – lcta0717 Oct 06 '21 at 08:19
@lcta0717 could you update your questions with both your new code and the output is being thrown? Because I couldn't understand what exactly is being returned as dict, as you've mentioned, and seeing the output, I might be able to help you. PS: Stick with the `find_elements_by_tag_name` by now, as it's the best way to go. – Luan Naufal Oct 06 '21 at 08:48
I'll update the new code, but I'm assuming that won't be the best way to go as I want to pick 1 iframe from 10 where ```display: 'block'``` is the style set – lcta0717 Oct 06 '21 at 17:39
@lcta0717 on my experience every time you deal with xpath directly, you'll be prone to error, since the website can change, you could write some small wrong details and break the logic, etc. If you have the 10 of them, you could either make a secondary query on the resulting list, or just loop each one of them until you find the one that has the `display: 'block'` property – Luan Naufal Oct 06 '21 at 20:15
That's the problem though, as listed above in option 2. I did find the right index that has the correct element. But ```find_elements_by_tag_name``` seems to be returning a ```list``` of ```dict```s, not an array or list of ```HTMLIFrameElement``` – lcta0717 Oct 07 '21 at 06:08
For reference, this is the page I'm trying to hit. Sometimes you have to refresh a few times to get the challenge, and it comes far more frequent when using selenium. Also, using headless mode causes the challenge to happen everytime... https://catalog.usmint.gov/coins/coin-programs/morgan-and-peace-silver-dollar-coins/ – lcta0717 Oct 08 '21 at 07:09

score 0 · Answer 2 · answered Sep 30 '21 at 12:57

0

¿Have you tried the Selenium documentation's example?

See cap. "Using an Index"

# switching to second iframe based on index
iframe = driver.find_elements_by_tag_name('iframe')[1]

# switch to selected iframe
driver.switch_to.frame(iframe)

answered Sep 30 '21 at 12:57

nugbe

139
1
5

1

Unfortunately, this doesn't work. Though I can see the iframe in the innerHTML of the parent, I can't find the iframe using either of your above methods – lcta0717 Oct 05 '21 at 13:07

score 0 · Answer 3 · answered Oct 04 '21 at 15:28

Whenever I encountered iframes I most of the time couldn't access them directly. Instead I had to change the xpath to the iframe its parent element and access it from there as shown below.

try:
    # instead of xpath = '//iframe' try accessing the parent element and add /iframe
    iframe_xpath = '//*[@id="parent-element"]/iframe'
    
    # or more specifically to get the right one in your case
    iframe_xpath = '//*[@id="parent-element"]/iframe[contains(@style, "display: block")]'

    WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, iframe_xpath)))
              

    driver.switch_to.frame(driver.find_element(By.XPATH, iframe_xpath))
    # Do your thing in the frame
    ...

    # go back to the normal content
    driver.switch_to.default_content()
except:
    # Ideally you catch each error separately; TimeoutException, NoSuchElementException, ...
    pass

Unfortunately, this doesn't work. In fact, the parent is showing that it has no children — lcta0717, Oct 05 '21 at 12:50
In your question you have a sample html where the parent element even has it's own class or am I misinterpreting this? Using your sample html, this would be the xpath to the iframe `iframe_xpath = '//*[@class="parent"]/iframe[contains(@style, "display: block")]'` — questioning, Oct 05 '21 at 15:27

Violet · Answer 4 · 2021-10-06T07:55:06.810

0

I've encountered this when the frames take time to render and are not caught initially by the code. The following approach has worked for me.

iframes = driver.find_elements_by_tag_name('iframe')

for iframe in iframes:
if 'block' in iframe.get_attribute('style'):
    driver.switch_to.frame(iframe)
    break

edited Oct 06 '21 at 07:55

answered Oct 05 '21 at 17:48

Violet

1
3

iframe element missing from selenium

4 Answers4

Extra Details