selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element [id=""]

Question

I'm trying to get the input tag and use click() by using selenium.

Here is my code:

#Try many ways to fix the error
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.select import Select 
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import selenium.webdriver.support.ui as ui
import time

driver = webdriver.Firefox()
wait = ui.WebDriverWait(driver, 10)
driver.get("""https://www.gso.gov.vn/px-web-2/?pxid=V0641&theme=N%C3%B4ng%2C%20l%C3%A2m%20nghi%E1%BB%87p%20v%C3%A0%20th%E1%BB%A7y%20s%E1%BA%A3n""")
driver.maximize_window() # For maximizing window

time.sleep(10)
selectButton= driver.find_element(By.ID,'footer')
print("select button : ", selectButton)
print("select button : ", selectButton.text)


time.sleep(3)
driver.close()

It worked fine and here the output:

select button :  <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="e94aba42-01a7-467b-81b5-16ce0b95dad8", element="32699e02-2621-4090-be18-2dfeac33b1c2")>
select button :  TRANG THÔNG TIN ĐIỆN TỬ TỔNG CỤC THỐNG KÊ
Bản quyền thuộc Tổng cục Thống kê
Địa chỉ: 54 Nguyễn Chí Thanh, Đống Đa, Hà Nội
Điện thoại: 024 73046666, máy lẻ 8668
Email: banbientap@gso.gov.vn
Ghi rõ nguồn trang Thông tin điện tử Tổng cục Thống kê (www.gso.gov.vn) khi trích lại thông tin từ địa chỉ này

And I change selectButton by replacing the id string to this:

selectButton= driver.find_element(By.ID,'ctl00_ContentPlaceHolderMain_VariableSelector1_VariableSelector1_VariableSelectorValueSelectRepeater_ctl01_VariableValueSelect_VariableValueSelect_SelectAllButton')

And I got the error:

Traceback (most recent call last):
  File "crawl.py", line 17, in <module>
    selectButton= driver.find_element(By.ID,'ctl00_ContentPlaceHolderMain_VariableSelector1_VariableSelector1_VariableSelectorValueSelectRepeater_ctl01_VariableValueSelect_VariableValueSelect_SelectAllButton')
  File "/home/dongky/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/home/dongky/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/dongky/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="ctl00_ContentPlaceHolderMain_VariableSelector1_VariableSelector1_VariableSelectorValueSelectRepeater_ctl01_VariableValueSelect_VariableValueSelect_SelectAllButton"]

I don't know why, anything wrong with the website's structure, or the id name is too long? Thanks for reading!

Answer 1

The element that you are looking for, is in iframe. So we would have to change the driver focus in order to interact with the desire element or elements :

Iframe xpath :

//iframe[contains(@src, 'https://pxweb.gso.gov.vn/pxweb/vi')]

and switch it like this :

WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(@src, 'https://pxweb.gso.gov.vn/pxweb/vi')]")))

and after this you can go ahead and do this :

selectButton= driver.find_element(By.ID,'ctl00_ContentPlaceHolderMain_VariableSelector1_VariableSelector1_VariableSelectorValueSelectRepeater_ctl01_VariableValueSelect_VariableValueSelect_SelectAllButton')

But I would suggest you to have a bit more reliable locator instead of the above ID.

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC