2

I am trying to navigate to each of the links and get their first news on the page.but can't get a unique xpath for the header info across all pages

Refer the attached pic, it shows 2 elements. how can i make it as unique to fetch only the header / title news. enter image description here

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
JeyanthiRanjit
  • 161
  • 1
  • 12

3 Answers3

2

To retrieve the header of the first news you need to induce WebDriverWait for the visibilityOfElementLocated() and you can use either of the following Locator Strategies:

  • Java and xpath and getText():

    System.out.println(new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[contains(@class, 'gs-u-display-inline-block@m')]//h3[@class='gs-c-promo-heading__title gel-paragon-bold nw-o-link-split__text']"))).getText());

  • Python and css_selector and text attribute:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[class*='gs-u-display-inline-block@m'] h3.gs-c-promo-heading__title.gel-paragon-bold.nw-o-link-split__text"))).text)

  • Console Output:

    Russia orders oldest rights group Memorial to shut

  • Note : For Python clients you have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • 1
    @JeyanthiRanjit Isn't your question _`it shows 2 elements. how can i make it as unique to fetch only the header / title news`_? – undetected Selenium Dec 28 '21 at 21:28
  • 1
    The layouts of none of the mentioned sites are identical, why do you feel a single shot [Locator Strategy](https://stackoverflow.com/questions/48054321/of-the-many-findelements-by-functions-in-selenium-when-would-you-use-one-over/48056120#48056120) will work across all of those sites? – undetected Selenium Dec 28 '21 at 21:37
  • That is a coding challenge I have been given with in an interview. If its not possible, will they give it as a question? But please advise if there is any strategy to achieve the result. – JeyanthiRanjit Dec 28 '21 at 21:43
  • 2
    @JeyanthiRanjit If you want a canonical answer based on multiple bbc.com sites, you may like to ask a new question. As per the description of this question _`it shows 2 elements. how can i make it as unique to fetch only the header / title news`_ I think you got well researched answers. – undetected Selenium Dec 28 '21 at 21:45
  • Thank you @DebanjanB – JeyanthiRanjit Dec 28 '21 at 21:54
1

You can select only the first element by using the parent's first child and then point to the child element, for example:

//div/div[1]/div/a/h3[@class="gs-c-promo-heading__title gel-paragon-bold nw-o-link-split__text"]

[1] is equal to :first-child

Oren Hahiashvili
  • 340
  • 1
  • 4
  • 6
  • Thanks, the challenge I am facing is while automating this process of navigating through each link from home page, I can only mention a common xpath to get header info across all pages. But its changing for every page. Can you help in sorting a common Xpath which can be used in an automation script to get the header across All link pages from Home page, when clicked one by one ? – JeyanthiRanjit Dec 28 '21 at 21:14
0

There are many headers on that page and they are changing. I mean some news are coming and being removed from this page.
If you want to get all the news title texts you can get all the headers elements and then iterate over them and extract their texts.
Something like this:

List<WebElement> headers = driver.findElements(By.xpath("//h3[@class='gs-c-promo-heading__title gel-pica-bold nw-o-link-split__text']"));
for (WebElement header : headers){    
    ((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", header);
    System.out.println(header.getText());
}
Prophet
  • 32,350
  • 22
  • 54
  • 79
  • Sure, but the thing is I can't find an xpath that can be common across all links from home page. the one you have mentioned is also not getting the header of all sublinks - sucha s NEWS has one kind of header and SPORTS has a diferent xpath for header, REEL & WORKLLIFE are even different Xpath for header info. THis is the challenge I am facing. – JeyanthiRanjit Dec 28 '21 at 21:11
  • 1
    As mentioned by DebanjanB we answered your question clearly. What you asking now is another question. We can try helping you with it, but you should accept some answer here, maybe at least upvote others, who were helpful, and ask another, new question for your new requirements. Please describe your self as clear as possible there to make us clearly understand what exactly you looking for. – Prophet Dec 28 '21 at 21:52