selenium - get element by position from iframe

Question

i have the following script that given x,y coordinates and a url returns the link of the element in these coordinates:

def get_link(url,x,y):
    DEFAULT_CHROME_WINDOWS_SIZE = '1024,2160'
    Options = ChromeOptions()
    Options.add_argument("--headless")
    Options.add_argument('--window-size={}'.format(DEFAULT_CHROME_WINDOWS_SIZE))
    driver = webdriver.Chrome(options=Options,executable_path="/usr/lib/chromium-browser/chromedriver")
    driver.get(url)
    time.sleep(3)
    elem_by_coords=driver.execute_script("return document.elementFromPoint(arguments[0], arguments[1])",
                                   x, y)
    link=elem_by_coords.get_attribute("href")
    return elem_by_coords.aria_role,link

for the following input:

elem_type,link=get_link("https://workflowy.com/s/aunt-sparkies-inc/JVbOUV2xLta97A77",448.5,413.5)

i get as expected:

elem_type=link
link=https://u.to/QM6CHA

However, for the following input:

elem_type,link=get_link("https://indd.adobe.com/view/72dacb55-647a-49ba-838b-8e82de10290a",482.5,1179.5)

i get this output:

elem_type=Iframe
link=None

the problem is that the link of the element is inside the iframe. i am certain that these coordinates match the element with the href link i want to extract but as i mentioned because it is in an iframe for some reason it takes the iframe and not the element inside the iframe..

so my question is how can i get the element at a certain x,y position that is inside the iframe?

score 0 · Answer 1 · answered Dec 23 '22 at 03:52

Since you are using document.elementFromPoint() that references a coordinate within an iframe, this method is expected to return the iframe object itself as per mdn docs:

If the element at the specified point belongs to another document (for example, the document of an <iframe>), that document's parent element is returned (the <iframe> itself)

As such, the function returned None, as iframe clearly doesn't have attribute href.

Having said that, as per SO thread, one could switch webdriver's focus to the iframe, and then access its elements. The coordinates will, however, need to be adjusted for the smaller iframe dimension.

For example, I will use Mozilla's iframe doc page and here is a screenshot:

If I want to access button CSS within iframe, we could do this:

def launch_url_switch_focus(url,x,y):
    chrome_srv = Service('./chromedriver')
    ch_options = webdriver.ChromeOptions()
    ch_options.add_argument("--start-maximized")
    driver = webdriver.Chrome(service=chrome_srv,options=ch_options)

    driver.get(url)
    # get and show main doc dimension
    doc_dim = driver.execute_script("doc=document.documentElement; return [doc.clientWidth,doc.clientHeight]")
    print('main doc dimension: ',doc_dim)

    demo_frame = driver.find_element(By.TAG_NAME, 'iframe')
    # switch focus to iframe
    driver.switch_to.frame(demo_frame)
    # get and show iframe dimension
    iframe_dim = driver.execute_script("doc=document.documentElement; return [doc.clientWidth,doc.clientHeight]")
    print('iframe dimension: ',iframe_dim)

    elem_by_coords=driver.execute_script("return document.elementFromPoint(arguments[0], arguments[1])",x, y)
    if elem_by_coords is not None:
        print(f'Found element: {elem_by_coords.text}')

Included code to show document dimension just to highlight the differences when switching focus, and the need to adjust the coordinates. Now, if we call the above function with URL and button CSS's relative coordinates, we get this:

>>> launch_url_switch_focus("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe",150,80)
main doc dimension:  [1920, 846]
iframe dimension:  [764, 419]
CSS

In your code, you could include a check if document.elementFromPoint() returns an iframe or a element, and then handle accordingly.

selenium - get element by position from iframe

1 Answers1