0

Using Selenium & Python I need to click on targets.simple.csv download a .csv file from this page: https://www.opensanctions.org/datasets/default/

Here is the code I have written:

import pandas as pd
import numpy as np
from datetime import date
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import time
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

chromedriver_path = r"./driver/chromedriver"
browser = webdriver.Chrome(executable_path=chromedriver_path)
url = "https://www.opensanctions.org/datasets/default/"
topics_xpath = '/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code'
browser.get(url)
browser.maximize_window()
time.sleep(10)  #Wait a little for page to load.
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]"))).click()
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code"))).click()

But I get this error:

ElementClickInterceptedException: element click intercepted: Element is not clickable at point (360, 1282)
  (Session info: chrome=114.0.5735.198)

However, the error does not occur if:

  • I run the script
  • As soon as the page opens, I manually scroll (with the mouse) where the downloadable csv file is.

If I do so, then the code downloads the .csv file.

Is there a way in Python to scroll to the file I want to download without me having to do it manually?

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Giampaolo Levorato
  • 1,055
  • 1
  • 8
  • 22

2 Answers2

1

you can refer this codes, the document is https://www.selenium.dev/documentation/webdriver/actions_api/wheel/#scroll-to-element

perform() can move down one by one, once it find the href, it will break the while loop. so you can use a little step one by one.

from_viewport() used to locate the first coordinate in the screen.

kkk_y = int(kkk.rect['y']) can get the element's y coordinate

below code can scroll to the element.
ActionChains(driver).scroll_from_origin(scroll_origin, 0, kkk_y).perform()

I don't install the chrome and driver, so I use the edge.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import time
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.actions.wheel_input import ScrollOrigin
from selenium.webdriver import ActionChains

driverpath = r"C:\Users\10696\Desktop\access\6\msedgedriver.exe"

service=Service(executable_path=driverpath)
driver = webdriver.Edge(service = service)
url = "https://www.opensanctions.org/datasets/default/"

topics_xpath = '/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code'
driver.get(url)
kk = driver.find_element(By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]")))
kk.click()

kkk = driver.find_element(By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")
scroll_origin = ScrollOrigin.from_viewport(0, 0)
kkk_y = int(kkk.rect['y'])
ActionChains(driver).scroll_from_origin(scroll_origin, 0, kkk_y).perform()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")))
time.sleep(2)
kkk.click()

another thing

you should get the website and use requests to download something, like below not using click, I can download this file normally targets.simple.csv. 


from selenium.webdriver.common.by import By
from selenium import webdriver
import os
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
import requests

driverpath = r"C:\Users\10696\Desktop\access\6\msedgedriver.exe"

service=Service(executable_path=driverpath)
driver = webdriver.Edge(service = service)
url = "https://www.opensanctions.org/datasets/default/"

driver.get(url)
kkk = driver.find_element(By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")
hrefkkk = kkk.get_attribute("href")
if hrefkkk:
    con = requests.get(hrefkkk)
    if con.status_code==200:
        with open(os.path.join(r'C:\Users\10696\Desktop\access\6', kkk.text), 'wb') as obj:
            obj.write(con.content)
Jiu_Zou
  • 463
  • 1
  • 4
1

To click on the clickable element you need to scroll down a few pixels and inducing WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using PARTIAL_LINK_TEXT:

    browser.get("https://www.opensanctions.org/datasets/default/")
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
browser.execute_script("scroll(0, 1000);") # Scroll Down
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "targets.simple.csv"))).click()

  • Using CSS_SELECTOR:

    browser.get("https://www.opensanctions.org/datasets/default/")
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
browser.execute_script("scroll(0, 1000);") # Scroll Down
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='targets.simple.csv'] > code"))).click()

  • Using XPATH:

    browser.get("https://www.opensanctions.org/datasets/default/")
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
browser.execute_script("scroll(0, 1000);") # Scroll Down
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a/code[text()='targets.simple.csv']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

  • Browser Snapshot:

opensanctions

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352