Unable to find element in Python

Question

I am trying to webscrape price list from a website and to do that I need to bypass age verification. To do that I am using following code to select the year "1999" but I am getting this error ----

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[name="year_field"]"}

The code that I use:

from selenium import webdriver

from selenium.webdriver.common.keys import Keys

import time

from selenium.webdriver.support.ui import Select

PATH="chromedriver.exe"


driver=webdriver.Chrome(PATH)

driver.get("https://valuebuds.com/pages/search-results-page")

driver.switch_to.frame("_hjRemoteVarsFrame")

time.sleep(5)

sel=Select(driver.find_element_by_name ("year_field"))

sel.Select_by_visible_text("1999")

print(driver.title)

Any help on this will be appreciated! Thanks

score 0 · Answer 1 · answered Feb 22 '22 at 20:31

The pop-up dialog is not inside a frame, so you should not switch to frame you are switching here.
Also you should use Expected Conditions explicit waits instead of hardcoded pauses.
And your locator is wrong, it is year_field id, not name attribute there.
This should work better:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys

import time

from selenium.webdriver.support.ui import Select

PATH="chromedriver.exe"


driver=webdriver.Chrome(PATH)
wait = WebDriverWait(driver, 20)

driver.get("https://valuebuds.com/pages/search-results-page")

wait.until(EC.presence_of_element_located((By.ID, "year_field")))
sel=Select(driver.find_element_by_id("year_field"))

sel.Select_by_visible_text("1999")

Thank you so much! However, when I try to run the above code, I receive the below error AttributeError Traceback (most recent call last) C:\Users\SHEKHA~1\AppData\Local\Temp/ipykernel_19120/340659611.py in 20 sel=Select(driver.find_element_by_id("year_field")) 21 ---> 22 sel.Select_by_visible_text("1999") AttributeError: 'Select' object has no attribute 'Select_by_visible_text' — Shekhar Singh, Feb 22 '22 at 20:58

undetected Selenium · Accepted Answer · 2022-02-22T21:18:14.320

The html-select element for the year isn't within the iframe. So you don't have to switch frames.

valuebuds_iframe

To select the year as 1999 using Selenium from the dropdown you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get("https://valuebuds.com/pages/search-results-page")
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#year_field")))).select_by_visible_text('1999')

Using XPATH:

driver.get("https://valuebuds.com/pages/search-results-page")
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='year_field']")))).select_by_visible_text('1999')

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Browser Snapshot:

valuebuds

References

You can find a couple of relevant discussions in:

Unable to find element in Python

The code that I use:

2 Answers2

References