The Python selenium code I have here takes us to a page with some datasets I'm interested in scraping. Here's my code.
import time, os
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
preferences= {"download.default_directory": os.getcwd(), "directory_upgrade": True}
options.add_experimental_option("prefs", preferences)
#options.headless = True
options.add_experimental_option('excludeSwitches', ['enable-logging'])
url = "http://www.ssp.sp.gov.br/transparenciassp/"
# Path of my WebDriver
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
wait = WebDriverWait(driver, 10)
# to maximize the browser window
driver.maximize_window()
#get method to launch the URL
driver.get(url)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#cphBody_btnHomicicio"))).click()
for x in range(18, 19):
year = '#cphBody_lkAno'+str(x)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, year))).click()
for x in range(1, 2):
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#cphBody_lkMes'+str(x)))).click()
So far this works, but my issue comes with the last loop of this syntax, even though it does exactly what I want. What I want to do, is select all of the elements on the page with a CSS selector which contain the text "cphBody_lkMes" and put them in a list like
['#cphBody_lkMes1','#cphBody_lkMes2', ...]
so that I can loop through them accordingly. The current code appears messy.
When I try
for x in range(18, 19):
year = '#cphBody_lkAno'+str(x)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, year))).click()
meses = driver.find_elements_by_css_selector("//*[contains(text(),'cphBody_lkMes')]")
meses
for x in range(1, 2):
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#cphBody_lkMes'+str(x)))).click()
the list comes up as empty, and an error says
An invalid or illegal selector was specified.
How might I solve this?