I can't parse the item. Selenium Python

Question

I do this. First I parsed the main block, and then some of the elements from it, but I get the same element. Here is the code:

driver.get("site")
time.sleep(3)

diary_main = driver.find_elements(By.XPATH, "//div[@class='diary-day']")
for i in diary_main:
    diary_date = i.find_element(By.XPATH, "//div[@class='diary-day-date']").get_attribute("data-date")
    print(diary_date)
    print(i.text)
    time.sleep(2)

diary_date outputs the same item Sample HTML:

<div class="diary-day">
     <div class="diary-day-date" data-date="Sunday"></div>
</div>
<div class="diary-day">
     <div class="diary-day-date" data-date="Monday"></div>
</div>
<div class="diary-day">
     <div class="diary-day-date" data-date="Tuesday"></div>
</div>
<div class="diary-day">
     <div class="diary-day-date" data-date="Wednesday"></div>
</div>

Update the question with the relevant HTML. – undetected Selenium Feb 23 '23 at 20:09 — undetected Selenium, Feb 23 '23 at 20:09

score 1 · Accepted Answer · answered Feb 23 '23 at 20:18

Probably problem with diary_main variable. It can't persist all the elements or timing issues. You can try this,

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get("site")

#wait for diary_main elements to be present
diary_main = WebDriverWait(driver, 5).until(EC.presence_of_all_elements_located((By.XPATH, "//div[@class='diary-day']")))

for i in diary_main:
    diary_date = i.find_element(By.XPATH, ".//div[@class='diary-day-date']").get_attribute("data-date")
    print(diary_date)
    print(i.text) 
    time.sleep(2)

Thanks for the help! I have a small addition, and if I want to continue searching for elements through `i`, what should I do? Through the normal option error — Deprool, Feb 23 '23 at 21:07

score 0 · Answer 2 · answered Feb 23 '23 at 22:57

To extract the values of the data-date attribute ideally you have to induce WebDriverWait for the visibility_of_all_elements_located() and using List Comprehension you can use either of the following locator strategies:

Using CSS_SELECTOR:

print([my_elem.get_attribute("data-date") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.diary-day > div.diary-day-date")))])

Using XPATH:

print([my_elem.get_attribute("data-date") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='diary-day']/div[@class='diary-day-date']")))])

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

I can't parse the item. Selenium Python

2 Answers2