Selenium web scraping iframe

Question

i want to read the toner values ​​on the web pages of the various printers in my office.

The problem is that the page is made up of several frames, and the one in which there is the remaining toner, is written in js and I can't read it even with selenium

This is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.expected_conditions import (
    presence_of_element_located)
from selenium.webdriver.support.wait import WebDriverWait

def get_comment_count(driver, url):
    driver.get(url)
    wait = WebDriverWait(driver, 3)
    e = driver.find_elements_by_xpath("/html/frameset/frame")
    driver.switch_to_frame(e[0])
    toner_iframe = driver.find_elements_by_xpath('//*[@id="contain"]')
    # iframe_url = toner_iframe.get_attribute('src')
    #driver.switch_to_frame(toner_iframe)
    driver.switch_to.frame(toner_iframe)
    print(toner_iframe)
    
url = "https://pritner_web_page"

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')

driver = webdriver.Chrome(options=options)

get_comment_count(driver,url)

I tried also...

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')

driver = webdriver.Chrome(options=options)
driver.get("http://printer_web_page")


WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'wlmframe')))

WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'toner')))
page_source=driver.page_source
print(page_source)

This is DOM Inspector of page. The various frames are dynamic and written in js as follows:

The code I wrote is just one of several different attempts to get to the frame, but to no avail

Answer 1

The element is within nested <frame> / <iframe> elements so you have to:

• Induce WebDriverWait for the parent frame to be available and switch to it.

• Induce WebDriverWait for the child frame to be available and switch to it.

• Induce WebDriverWait for the desired element to be clickable.

• You can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    driver.get("http://printer_web_page")
    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='wlmframe']")))
    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#toner[name='toner']")))
    

  • Using XPATH:

    driver.get("http://printer_web_page")
    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[@name='wlmframe']")))
    WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[@id='toner' and @name='toner']")))
    

• Note : You have to add the following imports :

   from selenium.webdriver.support.ui import WebDriverWait
   from selenium.webdriver.common.by import By
   from selenium.webdriver.support import expected_conditions as EC
  

---

Reference

You can find a couple of relevant discussions in:

• Ways to deal with #document under iframe
• Switch to an iframe through Selenium and python
• How to write appropriate Xpath to locate text value
• How To sign in to Applemusic With Python Using Chrome Driver With Selenium