How to access the iframe using Selenium and Python on Glassdoor

Question

I am trying to automate application process on Glassdoor using the EasyApply button. Now after identifying the EasyApply Button and successfully clicking it, I need to switch to the Frame so as to access the HTML content of the form to be able to send in my form details.

I used:

wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))

to perform the switching but still could not access the frame's html content.

Here is the block that performs this operation:

from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#if it has easy-apply, then perform application

if len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button')) > 0:
    driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button').click()
    wait = WebDriverWait(driver, 50)
    time.sleep(5)
    wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))
    name = driver.find_element(By.CSS_SELECTOR, '#input-applicant.name')
    name.send_keys('Oluyele Anthony')
elif len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a')) > 0:
    driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a').click()

Here is the HTML content for the frame

<iframe name="indeedapply-modal-preload-1659146630884-iframe" id="indeedapply-modal-preload-1659146630884-iframe" scrolling="no" frameborder="0" title="Job application form container" src="https://apply.indeed.com/indeedapply/xpc?v=5#%7B%22cn%22:%224AaHlXdnW4%22,%22ppu%22:%22https://www.glassdoor.com/robots.txt%22,%22lpu%22:%22https://apply.indeed.com/robots.txt%22,%22setupms%22:1659146630959,%22preload%22:true,%22iaUid%22:%221g96dgr7uii3h800%22,%22parentURL%22:%22https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox%22%7D" style="border: 0px; vertical-align: bottom; width: 100%; height: 100%;"></iframe>

Apparently, after running the cell, the:

TimeoutException: Message:

Error occurs which shows that the frame is not being switched to.

Here is the url as demanded.... https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox — OLUYELE SUNDAY ANTHONY, Jul 31 '22 at 23:54

score 1 · Accepted Answer · answered Jul 30 '22 at 08:54

The middle part of the value of id attribute i.e. 1658752913396 is dynamically generated and is bound to change sooner/later. They may change next time you access the application afresh or even while next application startup. So can't be used in locators.

Solution

To switch to the <iframe> you can use either of the following Locator Strategies:

Using id attribute:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][id^='indeedapply-modal-preload']")))

Using XPATH:

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@id, 'indeedapply-modal-preload')]")))

Using name attribute:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][name^='indeedapply-modal-preload']")))

Using XPATH:

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@name, 'indeedapply-modal-preload')]")))

Note : You have to add the following imports :

 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.common.by import By
 from selenium.webdriver.support import expected_conditions as EC

Thank you @undetected Selenium, this really helped me a lot, I believe I have been able to switch to the frame successfully, but now I am not getting access to the HTML content of the frame. It keeps throwing ```NoSuchElementException``` error. — OLUYELE SUNDAY ANTHONY, Aug 01 '22 at 00:38
@OLUYELESUNDAYANTHONY Sounds like a new question all together. Feel free to raise a new question with your new requirements. — undetected Selenium, Aug 01 '22 at 03:38

Barry the Platipus · Answer 2 · 2022-09-22T12:49:21.277

There are 2 iframes in that page and the second iframe contains another (nested) iframe. The following code will sort out your problem (setup is for linux, but you can figure it out - pay attention to imports, and to the code after getting the url):

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")

webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)

url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)

button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
button.click() 
iframes = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.TAG_NAME, "iframe")))
print(len(iframes))
for iframe in iframes:
    print(iframe.get_attribute('id'))
browser.switch_to.frame(iframes[1])
t.sleep(2)
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
t.sleep(2)
applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
applicant_name.click()
applicant_name.send_keys('hello dolly')

Let me know if it works for you.

UPDATE: It appears that website changed its structure since yesterday, and now there are 11/13 iframes per page. Also, if the job selected from the main column does not offer Easy Apply, you get an error. The updated code (below) is selecting the Easy Apply jobs from page, goes through each of them, clicks easyapply button, select the correct iframe/nested iframe, and does stuff to that form. It currently works, until that website will change its format again:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")

webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)

url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)

jobs = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//li[@data-is-easy-apply='true']")))

while True:
    for job in jobs:
        try:
            print(job.text)
            job.click()
            try:
                WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[@alt='Close']"))).click()
            except Exception as e:
                print('no request to login')

            t.sleep(2)
            button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
            button.click() 
            WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form container']")))
            t.sleep(2)
            WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
            t.sleep(2)
            applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
            applicant_name.click()
            applicant_name.send_keys('hello dolly') 
            t.sleep(2)
            WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='form-action-cancel']"))).click()
            t.sleep(2) 
            browser.switch_to.default_content()


            t.sleep(2)
        except Exception as e:
            print(e)
            break

Yeah, the id(s) are printed but still having a ```TimeoutException``` Error — OLUYELE SUNDAY ANTHONY, Aug 01 '22 at 20:38
I retested it from several IPs, and it works as intended. Again: are you running the code exactly as written above, or did you forget, for instance, to click first in the input, or to switch again to the nested iframe, once you're in the second iframe? — Barry the Platipus, Aug 01 '22 at 20:51
Upon re-checking, the website changed its structure, and also the first job did not offer EasyApply. I wrote a new logic, and now it should work - check the updated response. FYI - the original code was valid (for the previous webpage structure), I always test my responses several times before writing them. — Barry the Platipus, Aug 01 '22 at 22:22
Wow, worked like a magic... I think I now see what I am doing wrong, I should actually switch to the two nested frames, rather than switching only to the first frame. And also, the code incorporated the solution given initially by undetected Selenium using the locator strategy trying to access dynamically generated elements. Thank you @platipus_on_fire — OLUYELE SUNDAY ANTHONY, Aug 03 '22 at 11:04

How to access the iframe using Selenium and Python on Glassdoor

2 Answers2

Solution