Get href link with selenium (python)

Question

I want get all href from subelements. parent class is search-content it has parent divs card-col and in these divs there is another 1 div and then href. I want to get just this href link

this is my code-->

el=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CLASS_NAME, "search-content-cards")))
el_hrefs=el.find_elements_by_xpath(".//a[@href]")
for i in el_hrefs:
    print(i)

output is many elements

<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="f7c84bf8-c20c-4b70-8ba5-414e822bba21")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="1e078e8b-104f-4299-94b1-8741cf30f047")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="d8b4b5e0-6291-4fd2-ae04-faee245462d1")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="ef06e8ac-321c-40db-9f6c-40dd3a3b07de")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="e14cf667-1bf4-434c-b9a2-1c4f362398d2")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="8e549221-eca4-41cf-943d-3cb0f6f75d50")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="afd597fb-1bb0-48fb-8646-6c43cb17ab38")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="8f3a655e-d3cd-4748-934a-2c9000481ed3")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="a1706e30-fad0-4799-871f-c5a928c69009")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="156847c9-5d7f-4963-82fa-baaf2b8f6e7f")>
<selenium.webdriver.remote.webelement.WebElement (session="0a4b52d1575e427e34d6b790a284c501", element="99c320b4-f6f1-4eb4-abec-4be4df790b71")>

who can help me?

cruisepandey · Accepted Answer · 2022-01-13T17:18:12.367

1

i in your case is a web element, and to extract the .text, you should not just print i, it should be print(i.text).

Moreover if you want to extract the href off of the a tag, then you should use .get_attribute('href')

Secondly, I think you should use CSS_SELECTOR

div.search-content-cards

instead of CLASS_NAME

Also a tag is descendant.

so your effective code should look like this:

el = WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.search-content-cards")))
el_hrefs = el.find_elements_by_xpath(".//descendant::a[@href]")
for i in el_hrefs:
    print(i.get_attribute('href'))

edited Jan 13 '22 at 17:18

answered Jan 13 '22 at 17:11

cruisepandey

28,520
6
20
38

can u tell me what's mean div in head? (div.search-content-cards) i can't access with only class name? – kokiwebaaa Jan 13 '22 at 17:47
that div has a class (name having multiple spaces), if you use the class name, Selenium won't be able to find any element cause of spaces, you can remove the spaces and put `.` in it to make a CSS selector. – cruisepandey Jan 13 '22 at 17:55

score 1 · Answer 2 · answered Jan 13 '22 at 17:41

To extract the values of href attribute instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.search-content-cards a.d-block")))])

Using XPATH:

print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[contains(@class, 'search-content-cards')]//a[contains(@class, 'd-block')]")))])

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Get href link with selenium (python)

2 Answers2