Selenium can not find elements in dynamic web page, page source does not be loaded completely

Question

I try to recover some elements in a web page with selenium but the page_source I'm getting it does not have that elements loaded.

Find element returns elem.text empty and driver.page_source does not have the id titulotramitedocu.

What am I missing?

Code:

URL = "https://seu.conselldemallorca.net/fitxa?key=91913"
driver = webdriver.Chrome()
driver.get(URL)
try:
    driver.implicitly_wait(20)
    elem = driver.find_element(By.ID,"titulotramitedocu")
    print(elem.text)
finally:
    driver.quit()

I also tried with a wait..

element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "titulotramitedocu"))
)

Answer 1

To locate and print the text from the visible element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR:

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.titulotramitedocu#titulotramitedocu > h1"))).text)
  

• Using XPATH:

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='titulotramitedocu' and @id='titulotramitedocu']//h1"))).text)
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

• Console Output:

  Concurs de mèrits de personal funcionari del Consell de Mallorca per a la categoria d'enginyer-a tècnic-a industrial d'administració especial (codi CFCEA2/024)