How to iterate through the names of all the users from the webpage using Python and Selenium

Question

I am new to coding and need someone to hold my hand please...I am trying to create a bot that will use the nextdoor website to look for posts and send promotions to users through their messaging system. So far I have been able to perform login, search, and navigate to the Posts tab. After this point is where I am stuck. I will post a picture and also my code so you can see the issue. basically once I navigate to the posts tab, some posts appear from different users. I checked the html and looks like they have same div class. How do I make selenium check each users post. So far I am only able to find info on the first one and after that my code ends. I think I am having an iteration issue. Please help.

import sys
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

PATH =Service("C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=PATH)

driver.get("https://nextdoor.com/news_feed/")
print(driver.title)
#driver.quit()
time.sleep(3)

#Finds username and password id's and logs into web page
search = driver.find_element(By.ID, "id_email")
search.send_keys("private")
search = driver.find_element(By.ID, "id_password")
search.send_keys("private")
search = driver.find_element(By.ID, "signin_button").click()
time.sleep(3)

#In the next section of code we will execute searches
search = driver.find_element(By.ID, "search-input-field")
search.send_keys("remodel")
search.send_keys(Keys.RETURN)
time.sleep(3)
#This section will locate posts tab and click on it
search = driver.find_element(By.XPATH,"(//a[normalize-space()='Posts'])[1]").click()
time.sleep(2)
#This section will delete what was typed in the search bar
delete_search = driver.find_element(By.ID, "search-input-field")
delete_search.clear()
time.sleep(2)

for person in driver.find_elements(By.CLASS_NAME, "css-15luflj"):
    title = person.find_element(By.XPATH, "(//span[@data-testid='styled-text'] 
[normalize-space()='Miguel Med'])[1]").text

    persons.append({'username': title})


print(persons)

Page screenshot:

enter image description here

Element screenshot:

enter image description here

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. — Community, Aug 14 '22 at 22:56
@CarlosM239 At which line are you exactly stuck? Can you share with us a set of demo credentials to construct & test our solution for you? — undetected Selenium, Aug 14 '22 at 23:13

score 0 · Answer 1 · answered Aug 14 '22 at 23:20

As you are able to able to find info on the first one presumably the locator strategies are almost correct. You just need to modify the locator from specific to generic.

Solution

Rather looking out for the specific user Miguel Med as:

title = person.find_element(By.XPATH, "(//span[@data-testid='styled-text'][normalize-space()='Miguel Med'])[1]").text

Collect for all the users as:

title = person.find_element(By.XPATH, "(//span[@data-testid='styled-text'][text()])[1]").text

How to iterate through the names of all the users from the webpage using Python and Selenium

1 Answers1

Solution