How to find the href attributes using Selenium Python

Question

I tried many of the web solution but I can't find how to find the HREF.

One of my code:

driver = webdriver.Chrome(executable_path='C:\chromedriver.exe')
driver.get(("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing"))

time.sleep(20)
driver.execute_script("window.scrollTo(0,10000)")

elems = driver.find_elements(By.CSS_SELECTOR, ".c-card-search__title.ng-star-inserted [href]")
links = [elem.get_attribute('href') for elem in elems]

The web site: https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=10&srt=/project/contentUpdateDate:decreasing

The insepctor:

<a _ngcontent-vtw-c99="" class="c-card-search__title ng-star-inserted" href="/project/id/881603"> Graphene Flagship Core Project 3 </a>

undetected Selenium · Accepted Answer · 2022-04-25T14:36:11.820

The href attributes are within the following <a> nodes.

<a _ngcontent-jdx-c99="" class="c-card-search__title ng-star-inserted" href="/project/id/881603"> Graphene Flagship Core Project 3 </a>

Solution

To print the values of the href attribute you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.c-card-search__title.ng-star-inserted[href]")))])

Using XPATH:

driver.get("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='c-card-search__title ng-star-inserted' and @href]")))])

Console Output:

['https://cordis.europa.eu/project/id/881603', 'https://cordis.europa.eu/project/id/945539', 'https://cordis.europa.eu/project/id/964827', 'https://cordis.europa.eu/project/id/956396', 'https://cordis.europa.eu/project/id/956544', 'https://cordis.europa.eu/project/id/101037031', 'https://cordis.europa.eu/project/id/101025100', 'https://cordis.europa.eu/project/id/101007165', 'https://cordis.europa.eu/project/id/959234', 'https://cordis.europa.eu/project/id/952520', 'https://cordis.europa.eu/project/id/959879', 'https://cordis.europa.eu/project/id/101005177', 'https://cordis.europa.eu/project/id/101004714', 'https://cordis.europa.eu/project/id/946044', 'https://cordis.europa.eu/project/id/961989', 'https://cordis.europa.eu/project/id/962483', 'https://cordis.europa.eu/project/id/862011', 'https://cordis.europa.eu/project/id/891402', 'https://cordis.europa.eu/project/id/886365', 'https://cordis.europa.eu/project/id/870378', 'https://cordis.europa.eu/project/id/870743', 'https://cordis.europa.eu/project/id/834195', 'https://cordis.europa.eu/project/id/852985', 'https://cordis.europa.eu/project/id/814837', 'https://cordis.europa.eu/project/id/861278', 'https://cordis.europa.eu/project/id/842243', 'https://cordis.europa.eu/project/id/833507', 'https://cordis.europa.eu/project/id/825496', 'https://cordis.europa.eu/project/id/820350', 'https://cordis.europa.eu/project/id/820445', 'https://cordis.europa.eu/project/id/786890', 'https://cordis.europa.eu/project/id/824295', 'https://cordis.europa.eu/project/id/769086', 'https://cordis.europa.eu/project/id/768960', 'https://cordis.europa.eu/project/id/772337', 'https://cordis.europa.eu/project/id/793965', 'https://cordis.europa.eu/project/id/797942', 'https://cordis.europa.eu/project/id/757184', 'https://cordis.europa.eu/project/id/639020', 'https://cordis.europa.eu/project/id/694665', 'https://cordis.europa.eu/project/id/679403', 'https://cordis.europa.eu/project/id/677061', 'https://cordis.europa.eu/project/id/677493', 'https://cordis.europa.eu/project/id/679843', 'https://cordis.europa.eu/project/id/677823', 'https://cordis.europa.eu/project/id/716472', 'https://cordis.europa.eu/project/id/721537', 'https://cordis.europa.eu/project/id/725627', 'https://cordis.europa.eu/project/id/694343', 'https://cordis.europa.eu/project/id/677650', 'https://cordis.europa.eu/project/id/678423', 'https://cordis.europa.eu/project/id/679243', 'https://cordis.europa.eu/project/id/846617', 'https://cordis.europa.eu/project/id/101002761', 'https://cordis.europa.eu/project/id/848325', 'https://cordis.europa.eu/project/id/850488', 'https://cordis.europa.eu/project/id/854931', 'https://cordis.europa.eu/project/id/101027737', 'https://cordis.europa.eu/project/id/101030066', 'https://cordis.europa.eu/project/id/101020016', 'https://cordis.europa.eu/project/id/101024758', 'https://cordis.europa.eu/project/id/850769', 'https://cordis.europa.eu/project/id/863227', 'https://cordis.europa.eu/project/id/861910', 'https://cordis.europa.eu/project/id/878074', 'https://cordis.europa.eu/project/id/875905', 'https://cordis.europa.eu/project/id/862111', 'https://cordis.europa.eu/project/id/846793', 'https://cordis.europa.eu/project/id/862311', 'https://cordis.europa.eu/project/id/101003472', 'https://cordis.europa.eu/project/id/101036910', 'https://cordis.europa.eu/project/id/101036449', 'https://cordis.europa.eu/project/id/101008571', 'https://cordis.europa.eu/project/id/101018420', 'https://cordis.europa.eu/project/id/101037564', 'https://cordis.europa.eu/project/id/101000622', 'https://cordis.europa.eu/project/id/101021936', 'https://cordis.europa.eu/project/id/101037424', 'https://cordis.europa.eu/project/id/101008233', 'https://cordis.europa.eu/project/id/101003470', 'https://cordis.europa.eu/project/id/101008297', 'https://cordis.europa.eu/project/id/101003815', 'https://cordis.europa.eu/project/id/956355', 'https://cordis.europa.eu/project/id/101003914', 'https://cordis.europa.eu/project/id/101017304', 'https://cordis.europa.eu/project/id/101007326', 'https://cordis.europa.eu/project/id/101003491', 'https://cordis.europa.eu/project/id/101033819', 'https://cordis.europa.eu/project/id/101016902', 'https://cordis.europa.eu/project/id/101003826', 'https://cordis.europa.eu/project/id/955317', 'https://cordis.europa.eu/project/id/101004633', 'https://cordis.europa.eu/project/id/872427', 'https://cordis.europa.eu/project/id/101006249', 'https://cordis.europa.eu/project/id/101005122', 'https://cordis.europa.eu/project/id/952851', 'https://cordis.europa.eu/project/id/855187', 'https://cordis.europa.eu/project/id/892429', 'https://cordis.europa.eu/project/id/865067', 'https://cordis.europa.eu/project/id/883520']

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

OK that works. It's a bit complicated for me sorry. I understood the usefulness of the wait but I do not understand the usefulness of this part 'WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()' there, nor how you find the link inside. — rocheteau, Apr 25 '22 at 13:07
Let's discuss the issue in [Selenium](https://chat.stackoverflow.com/rooms/223360/selenium) room. — undetected Selenium, Apr 25 '22 at 13:08

score 0 · Answer 2 · answered Apr 25 '22 at 12:34

0

I hope thi swill solve your question, also, please let me if this will not work for you.

elems = driver.find_elements(By.XPATH,"//a[@href]")
for elem in elems:
    print(elem.get_attribute("href"))

answered Apr 25 '22 at 12:34

Akzy

1,817
1
7
19

Yes I tried this one but i have all the href in the page not just those who look like this one '/project/id/881603' – rocheteau Apr 25 '22 at 12:38

How to find the href attributes using Selenium Python

2 Answers2

Solution