Selenium - find article by h1 and p text

Question

How to find article on website by h1 and p text like on image below?

I tried this, where I can found all articles and I don't know how to find this one with text in h1 and by text in p. And then I would like to click on this.

text = driver.find_elements_by_xpath("//article/div[contains(@class,'inner-article')]/h1")

What would you like to do then? if you want to extract stuff from it, you might need some scraping library, if you'd like to click, please specify what — Timeler, Nov 30 '20 at 08:58
yes the url please, and would you like to click on the beanie custom first or the Red? — Timeler, Nov 30 '20 at 09:00
@Filip cant seem to find it, where on the website is that button? — Timeler, Nov 30 '20 at 09:13

score 2 · Answer 1 · answered Nov 30 '20 at 11:01

To extract and print the text Beanie Custom First and Red you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR and text attribute:

Printing Beanie Custom First:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "article.inner-article h1 > a.name-link[href='/shop/asd']"))).text)

Printing Red:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "article.inner-article p > a.name-link[href='/shop/asd']"))).text)

Using XPATH and get_attribute():

Printing Beanie Custom First:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//article[@class='inner-article']//h1/a[@class='name-link' and @href='/shop/asd']"))).get_attribute("innerHTML"))

Printing Red:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//article[@class='inner-article']//p/a[@class='name-link' and @href='/shop/asd']"))).get_attribute("innerHTML"))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

Outro

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

score 1 · Accepted Answer · answered Nov 30 '20 at 10:07

text = driver.find_elements_by_xpath("//article/div[contains(@class,'inner-article')][h1/a[contains(text(),"Beanie")]][p/a[contains(text(),"Red")]]")

you can use above xpath, which will check whehter the parent element article/div has child elements h1/a and p/a with texts Beanie and Red respectively

in w3chool html editor is inside iframe so switch to iframe in your seelnium tests before tryng to find the element

Selenium - find article by h1 and p text

2 Answers2

Outro