Python Selenium Cannot find the href when it is on the website

Question

I have difficulty in finding the social media links on this particular website: https://www.digitalrealty.com/

The code I used:

url = 'https://www.digitalrealty.com/'
driver.get('https://www.digitalrealty.com/')
fb_link = driver.find_element_by_css_selector("a[href*='facebook.com']").get_attribute("href")

After I run the code, however, python returns "NoSuchElementException". Actually, when I look into the website and scroll down, the social media accounts are just at the bottom. I am not sure whether it is because the page is not loaded completely? But isn't .get() by default waiting for the page to load?

I am still new to handling these errors. Any help would be appreciated. Thanks in advance!

Answer 1

The website loads content on scroll, that's why to get to the social media accounts you need to scroll to the bottom of the page.
To wait for some conditions you have to use WebDriverWait and expected_conditions, more details about waits you can find here.

Here your code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 5)

url = "https://www.digitalrealty.com/"

driver.get(url)

#Scroll to the bottom
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

#Wait until presence of the facebook element. Presence of element=exist in the DOM, but can be not visible like here.
facebook = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "a[href*=facebook]")))

#Scroll to element will make element visible and we'll able to click or get text, attribute from it
driver.execute_script("arguments[0].scrollIntoView(true);", facebook)

#Wait for facebook element to be visible and click on it.
fb = wait.until(EC.visibility_of(facebook)).get_attribute("href")
print(fb)