1

for some reason when I try to read the xpath while web scraping it returns []. See the code below:

from selenium import webdriver
import time

PATH = 'D:\webdriver\chromedriver.exe'

driver = webdriver.Chrome(PATH)

url2 = 'https://www.rad.cvm.gov.br/ENET/frmGerenciaPaginaFRE.aspx?NumeroSequencialDocumento={}&CodigoTipoInstituicao=1'.format('91514')

driver.get(url2)

time.sleep(2)

variavel2 = driver.find_elements_by_xpath('//*[@id="ctl00_cphPopUp_tbDados"]/tbody/tr[2]/td[3]')

print(variavel2)
DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Rapha
  • 13
  • 2
  • hi, interesting, perhaps [selenium waits](https://stackoverflow.com/a/27603477/11746212) might be of interest https://stackoverflow.com/questions/27600906/python-selenium-how-to-wait-before-clicking-on-link https://stackoverflow.com/questions/47251939/wait-until-button-is-clicked-in-selenium-webdriver-to-click-on-next-button – IronMan Nov 19 '20 at 00:57
  • @IronMan Hi I tried to use the WebDriverWait but it raises TimeoutException error – Rapha Nov 19 '20 at 01:25
  • @Rapha Your element target inside a frame. – frianH Nov 19 '20 at 03:27
  • it's an iframe. See: https://www.techbeamers.com/switch-between-iframes-selenium-python/ – DMart Nov 19 '20 at 04:29

2 Answers2

2

The page uses an iframe so you need to switch to the iframe.

Try this:

url2 = 'https://www.rad.cvm.gov.br/ENET/frmGerenciaPaginaFRE.aspx?NumeroSequencialDocumento={}&CodigoTipoInstituicao=1'.format('91514')

driver.get(url2)

time.sleep(2)
driver.switch_to.frame("iFrameFormulariosFilho")
variavel2 = driver.find_elements_by_xpath('//*[@id="ctl00_cphPopUp_tbDados"]/tbody/tr[2]/td[3]')

print(variavel2)
DMart
  • 2,401
  • 1
  • 14
  • 19
1

The table seems to load a little bit later, so you will need waits:

element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH,'//*[@id="ctl00_cphPopUp_tbDados"]/tbody/tr[2]/td[3]')))
Wasif
  • 14,755
  • 3
  • 14
  • 34