Unable to scrape text using By.CLASS_NAME in selenium

Question

Every time I try to scrape the HTML of a page I get the same error:

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified

I know that selenium changed the way of selecting an object by class name so I use the following code:

from selenium import webdriver

driver = webdriver.Chrome("C:\\Users\\PC\\Downloads\\chromedriver_win32\\chromedriver.exe")
url = "https://www.oddsportal.com/football/england/premier-league/"
driver.get(url)
element1 = driver.find_elements(By.CLASS_NAME, "flex items-center gap-1 my-1 align-center w-[100%]")
print(element1)
driver.quit()

I tried using "class name" instead of By.CLASS_NAME

In the HTML DOM, the `class` attribute is a **space-separated list** of class names. In your Selenium you use only one of those. — SiKing, Mar 17 '23 at 15:58
Refer this - https://stackoverflow.com/questions/47660523/selenium-python-finding-elements-by-class-name-with-spaces — Shawn, Mar 17 '23 at 16:09
Does this answer your question? [Selenium Python - Finding Elements by Class Name With Spaces](https://stackoverflow.com/questions/47660523/selenium-python-finding-elements-by-class-name-with-spaces) — SiKing, Mar 17 '23 at 16:51

score 0 · Answer 1 · answered Mar 17 '23 at 17:07

0

You can use below Css Selector , those spaces are multiple classes

By.cssSelector(".flex.items-center.gap-1.my-1.align-center.w-[100%]")

answered Mar 17 '23 at 17:07

Abhay Chaudhary

1,763
1
8
13

undetected Selenium · Answer 2 · 2023-03-18T00:05:15.777

This error message...

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified

...implies that selector you have used isn't a valid selector.

Solution

To extract the desired text ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get("https://www.oddsportal.com/football/england/premier-league/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.flex.items-center.gap-1.my-1.align-center > div > div"))).text)

Using XPATH:

driver.get("https://www.oddsportal.com/football/england/premier-league/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(@class, 'flex items-center gap-1 my-1 align-center')]/div/div"))).text)

Console Output:
```
Nottingham
1
:
2
Newcastle
```

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

Unable to scrape text using By.CLASS_NAME in selenium

2 Answers2

Solution