0

I'm trying to crawl data from this website https://www.topcv.vn/cong-ty/fpt-software/3.html, and I want to move to next page using Selenium. But when I use click() method, it just doesn't work.

<a href="javascript:void(0)" class="btn btn-paginate btn-next" data-page="1" data-id="11111"> <i class="fa-solid fa-chevron-right"></i> </a>

What do I do wrong?

I try using click():

driver.find_element(By.CLASS_NAME, 'btn btn-paginate btn-next').click()

but it just return on my command screen this: ` Traceback (most recent call last): File "e:\DataPre-handle\f1.py", line 20, in <module> driver.find_element(By.CLASS_NAME, 'btn btn-paginate btn-next').click()

`

  • The `class` attribute in the HTML is a **space-separated list** of classes. You use only one of those. – SiKing Jun 29 '23 at 16:01
  • Does this answer your question? [Find by Class name command is not working](https://stackoverflow.com/questions/57766234/find-by-class-name-command-is-not-working) – SiKing Jun 29 '23 at 16:02

3 Answers3

0

If you're trying to click that Next button, use: "a.btn-next" (By.CSS_SELECTOR) for your selector.

If you're looking for a reusable method to help you determine if an element is clickable:

def is_element_clickable(driver, selector, by="css selector"):
    """
    Returns whether the specified element selector is clickable.
    @Params
    driver - the webdriver object (required)
    selector - the locator for identifying the page element (required)
    by - the type of selector being used (Default: "css selector")
    @Returns
    Boolean (is element clickable)
    """
    try:
        element = driver.find_element(by=by, value=selector)
        if element.is_displayed() and element.is_enabled():
            return True
        return False
    except Exception:
        return False
Michael Mintz
  • 9,007
  • 6
  • 31
  • 48
0

As per the cocumentation of By implementation By.CLASS_NAME accepts only one classname. Passing multiple classnames will result into invalid selector.


Given the HTML:

<a href="javascript:void(0)" class="btn btn-paginate btn-next" data-page="1" data-id="11111"> 
    <i class="fa-solid fa-chevron-right"></i> 
</a>

To click on the element to move to next page as the desired element is a JavaScript enabled element, you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn.btn-paginate.btn-next > i"))).click()
    
  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn btn-paginate btn-next']/i"))).click()
    
  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
-1

You can use Selenium's waits. Below code will wait for 30s to check if the element is clickable(basically it will check if its visible and enabled).

WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn btn-paginate btn-next']"))).click()

Or this:

WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "(//a[@data-id='11111'])[2]"))).click()

Imports required:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Full working code:

driver = webdriver.Chrome()
driver.get("https://www.topcv.vn/cong-ty/fpt-software/3.html")
driver.maximize_window()
wait = WebDriverWait(driver, 30)

# below line will close the pop-up which appears as soon the webpage is opened
wait.until(EC.element_to_be_clickable((By.XPATH, "//i[@class='fa-solid fa-xmark']"))).click()
# below line will click on the next page button 
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn btn-paginate btn-next']"))).click()
Shawn
  • 4,064
  • 2
  • 11
  • 23
  • `(By.CLASS_NAME, 'btn btn-paginate btn-next')` will never work, no matter how long you wait. – SiKing Jun 29 '23 at 16:20
  • Yes you are right. `By.CLASS_NAME` will not work. As per question's title, I was emphasizing more on `element_to_be_clickable` feature. However, I had provided another solution too using XPATH strategy which will work. I have updated the answer and removed `CLASS_NAME` locator stretegy now. – Shawn Jun 29 '23 at 16:42