Selenium Using Python - I couldn't take elements by CSS_SELECTOR

Question

I want to select a list of items using a CSS selector. However, when I try to retrieve the elements on the page, I am only getting three items, even though there should be 50 items. After inspecting the page structure, I can confirm that it should contain all 50 elements. Nevertheless, I am still getting a list with only three elements. Is there something wrong with my code or css syntax?

product_names must be an iterable object with 50 items

product_names = driver.find_elements(By.CSS_SELECTOR, ".elements-title-normal")
time.sleep(1)
list1 = [] 
for i in product_names:
    list1.append(i.find_element(By.CSS_SELECTOR, ".elements-title-normal.one-line").text())
    time.sleep(1)
print(list1)

I got 3 items from "product_names"

And what would be the url of that page? – Barry the Platipus Jul 21 '23 at 10:27 — Barry the Platipus, Jul 21 '23 at 10:27

score 0 · Answer 1 · answered Jul 21 '23 at 22:53

To extract and print the texts ideally you need to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR and get_attribute("innerHTML"):

print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".elements-title-normal .elements-title-normal.one-line")))])

Using XPATH and text attribute:

print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//*[@class='lements-title-normal']//*[@class='elements-title-normal one-line']")))])

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Outro

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

Selenium Using Python - I couldn't take elements by CSS_SELECTOR

1 Answers1

Outro