Unable to get element error using Selenium in Python

Question

This is a weird one, I am trying to get product names and price from a website using selenium in Python. I am able to get the price but getting the unable to locate element error for product name.

I am using very similar code for both these items :

s3 = driver.find_element(By.CSS_SELECTOR, "span.weight-tile__PriceText-otzu8j-5.hJFddt")

print(s3.text)

s = driver.find_element(By.CSS_SELECTOR, "span.mobile-product-list-item__ProductName-zxgt1n-6.hBmGOx")

print(s.text)

Show us the html code for the elements you expect to find. Don't make us click on a website link. — John Gordon, Jun 27 '23 at 17:16
Please don't post *images* of text; just include the text itself (formatted as a code sample when appropriate). — larsks, Jun 27 '23 at 17:29
XPATH - //div[@class="desktop-product-list-item__Container-sc-8wto4u-0 jTzrhU"]//div[@class="desktop-product-list-item__ProductNameContainer-sc-8wto4u-12 bMVVKM"]//span — Samira Kumar Nanda, Jun 27 '23 at 17:29

score 1 · Accepted Answer · answered Jun 27 '23 at 17:28

css selector is trying to find the class which is valid for mobile size. If you want to find the element, make sure that the browser has mobile dimensions. Or if you want to catch the titles in the desktop version, in your case it is

s = driver.find_element(By.CSS_SELECTOR, "span.desktop-product-list-item__ProductName-sc-8wto4u-7.kjymBK")

score 1 · Answer 2 · answered Jun 27 '23 at 18:20

The classname attribute values like hJFddt, hBmGOx are dynamically generated and is bound to change sooner/later. They may change next time you access the application afresh or even while next application startup. So can't be used in locators.

Solution

To extract the prices you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get('https://https://dutchie.com/embedded-menu/the-circle/products/flower?)
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span[class^='desktop-product-list-item__ProductName']")))])

Using XPATH:

driver.get('https://https://dutchie.com/embedded-menu/the-circle/products/flower?)
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[starts-with(@class, 'desktop-product-list-item__ProductName')]")))])

Console Output:

['Cannabiotix | The Silk | 3.5g', 'HEX | Lemon Cherry Gelato | 3.5g', 'A Golden State | Garlic Blossom | 3.5g', 'Alien Labs | OZ Kush | 3.5g']

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Unable to get element error using Selenium in Python

2 Answers2

Solution