How Can I extract class value and save in csv using selenium?

Question

How Can I extract the value using selenium and python, My end goal is to store this value in a csv.

What I have tried:

#element=  driver.find_element_by_xpath("//*[@class='rt-tr-group']")
elements = driver.find_elements_by_class_name("product-form__price")
for value in elements:
    print(value.text)

But this returns an empty list?

HTML

Please let me know if my answer resolved your problem or still not? — Prophet, Dec 16 '21 at 09:01

score 0 · Answer 1 · answered Dec 16 '21 at 08:51

0

Looks like you are missing a wait / delay.
Try this

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".product-form__price")))
time.sleep(0.5)
elements = driver.find_elements_by_class_name("product-form__price")
for value in elements:
    print(value.text)

You will need these imports:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

And to initialize the wait object with

wait = WebDriverWait(driver, 20)

answered Dec 16 '21 at 08:51

Prophet

32,350
22
54
79

Getting a time out exception for this particular selector, even though everthing above is working. Strange – Dipen Ganguly Dec 17 '21 at 01:47
Can you share a link to that page? – Prophet Dec 17 '21 at 08:07

score 0 · Answer 2 · answered Dec 16 '21 at 14:27

To print the innertexts e.g. $53.37 you need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CLASS_NAME and get_attribute("innerHTML"):

print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CLASS_NAME, "product-form__price")))])

Using CSS_SELECTOR and get_attribute("textContent"):

print([my_elem.get_attribute("textContent") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span.product-form__price")))])

Using XPATH and text attribute:

print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[@class='product-form__price']")))])

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

Getting a timeout Exception in all three of the methods. any idea what might be causing this? — Dipen Ganguly, Dec 17 '21 at 01:42

How Can I extract class value and save in csv using selenium?

2 Answers2

References