0

I am trying to get the value #2011 which is a URL text from the HTML below. I tried the below code but didnt work. It says it is unable to locate the class

driver.find_element(By.XPATH, '//span[@class = "data-issue-and-pr-hovercards-enabled"]').get_attribute('a')

Can anyone help to correct the mistake? I am new to selenium.

<span data-issue-and-pr-hovercards-enabled>
  <span><span> · Fixed by <a href="https://github.com/mlpack/mlpack/pull/2011" data-hydro-click="{&quot;event_type&quot;:&quot;issue_cross_references.click&quot;,&quot;payload&quot;:{&quot;reference_location&quot;:&quot;ISSUE_HEADER&quot;,&quot;user_id&quot;:8344482,&quot;issue_id&quot;:490664092,&quot;pull_request_id&quot;:315249262,&quot;originating_url&quot;:&quot;https://github.com/mlpack/mlpack/issues/2008&quot;}}" data-hydro-click-hmac="aadfd36202d72a8e9a7ce379994bac18a7e2052adfb12d43eea1c8f41e12bfde" data-hovercard-type="pull_request" data-hovercard-url="/mlpack/mlpack/pull/2011/hovercard">#2011</a></span><span></span></span>
</span>

Here is the link to the website - github.com/mlpack/mlpack/issues/2008 I want to get the #2011 which is next to the Fixed by Text (below the title of the issue). Is it possible to do this?

Mano Haran
  • 634
  • 1
  • 6
  • 19

2 Answers2

1

In the given HTML data-issue-and-pr-hovercards-enabled is an attribute but not the value of class.

To extract the text #2011 ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR and text attribute:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span[data-issue-and-pr-hovercards-enabled] a[data-hovercard-type='pull_request'][data-hovercard-url='/mlpack/mlpack/pull/2011/hovercard']"))).text)

  • Using XPATH and get_attribute("innerHTML"):

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[@data-issue-and-pr-hovercards-enabled]//a[@data-hovercard-type='pull_request' and @data-hovercard-url='/mlpack/mlpack/pull/2011/hovercard']"))).get_attribute("innerHTML"))

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I am not sure whether this is what I am looking for. Here is the link to the website - https://github.com/mlpack/mlpack/issues/2008 I want to get the #2011 which is next to the Fixed by Text (below the title of the issue). Is it possible to do this? – Mano Haran Jan 30 '23 at 17:45
  • There are 2 similar instances of the text `#2011`, _Fixed by #2011_ and _closed this as completed in #2011_. Which one are you looking at? – undetected Selenium Jan 30 '23 at 18:00
  • Its the first one on the top. – Mano Haran Jan 30 '23 at 18:05
  • Thanks bro. The above comment worked - //span[contains(text(),"Fixed by")]//a – Mano Haran Jan 30 '23 at 18:11
1

Try the below XPath: This relative xpath will search for all tag names(*) which contains the text "#2011"

//*[contains(text(),'#2011')]

Or try the below one: Very similar explanation as above but this will search only within <a> tag

//a[contains(text(),'#2011')]

Update:

Try the below XPath:

//span[contains(text(),'Fixed by')]//a

Use .text method to fetch the required value. This will get you the below text value. enter image description here

Shawn
  • 4,064
  • 2
  • 11
  • 23