Selenium how to locate all elements with the tag h2

Question

Hello so given that i have navigated to this site

https://uk.trustpilot.com/categories

I would like to retrieve the headings of each catogory by using inspect element, I noticed that every heading of the catorgories has a h2 tag

My thoughts were to use a EC condition to wait till for the visibility of all elements that have a h2 tag which should return a list containing all h2 tags which i can then just call getattribute(text) on to get the text

def select_catogory(self):
        list = self.wait.until(EC.presence_of_all_elements_located((By.TAG_NAME, 'h2')))
        print(len(list))
        catogories = self.driver.find_elements(By.TAG_NAME,value ="h2")
        for cat in catogories:
            print(cat.get_attribute('text')

both presence_of_all_elements_located returned nothing and same with find_elements

)

I realised that my main issue was in my loop where i never had a break condition so it was giving a stale element warning, which i thought was caused by this function, Thanks for the help

def select_button_by_name(self, name_of_button: str):
        """
        Navigates to the button in question and clicks on it

        input: name_of_button (string)
        """
        self.wait
       
        buttons_by_href = self.driver.find_elements(By.XPATH,value ="//a[@href]") #finds all the href tags on the webpage
        for button in buttons_by_href:
           # Matches the attribute text to the provided button string
            if button.get_attribute("text") == name_of_button:
                #clicks on the button
                button.click()
                break

score 0 · Accepted Answer · answered Jan 20 '23 at 20:51

This SeleniumBase solution worked:

pip install seleniumbase, then run with python or pytest:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__, "-s")

class MyTestClass(BaseCase):
    def test_base(self):
        self.open("https://uk.trustpilot.com/categories")
        h2_elements = self.find_visible_elements("h2")
        print()
        for element in h2_elements:
            print(element.text.strip())

Output was:

Animals & Pets
Beauty & Well-being
Business Services
Construction & Manufacturing
Education & Training
Electronics & Technology
Events & Entertainment
Food, Beverages & Tobacco
Health & Medical
Hobbies & Crafts
Home & Garden
Home Services
Legal Services & Government
Media & Publishing
Money & Insurance
Public & Local Services
Restaurants & Bars
Shopping & Fashion
Sports
Travel & Vacation
Utilities
Vehicles & Transportation

Yeah it worked for me i was being a idiot, in my previous step I had a button click within a for loop which i never exited out of so it was giving a stale element error, i fixed it by breaking out of the loop — Darman, Jan 20 '23 at 21:00

undetected Selenium · Answer 2 · 2023-01-20T22:07:07.943

You can extract the headings of each catogory using Selenium and python in a single line of code inducing WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR and get_attribute("textContent"):

driver.get('https://uk.trustpilot.com/categories')
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
print([my_elem.get_attribute("textContent") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "h2[class^='typography_heading']")))])

Using XPATH and text attribute:

driver.get('https://uk.trustpilot.com/categories')
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
print([my_elem.text for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.XPATH, "//h2[text()]")))])

Console Output:

['Animals & Pets', 'Beauty & Well-being', 'Business Services', 'Construction & Manufacturing', 'Education & Training', 'Electronics & Technology', 'Events & Entertainment', 'Food, Beverages & Tobacco', 'Health & Medical', 'Hobbies & Crafts', 'Home & Garden', 'Home Services', 'Legal Services & Government', 'Media & Publishing', 'Money & Insurance', 'Public & Local Services', 'Restaurants & Bars', 'Shopping & Fashion', 'Sports', 'Travel & Vacation', 'Utilities', 'Vehicles & Transportation']

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Selenium how to locate all elements with the tag h2

2 Answers2

Linked