How to locate the LOAD MORE button using Selenium Python

Question

I'm having trouble trying to figure out how to locate the path for a "Load More Button" on this site I am trying to scrape.

https://www.ufc.com/athletes/all

The button html:

<a class="button" href="?gender=All&amp;search=&amp;page=1" title="Load more items" 
 rel="next">Load More</a>  
 "Load More" == $0
 ::after == 0$
 

Answer 1

You didn't provide any attempts and I can see 3 right off the top of my head that should work...

1. CSS selector, a[title='Load more items']
2. XPath, //a[text()='Load More']
3. By link text, "Load More"

In general you should prefer CSS selectors because they are more widely supported, faster, and have simpler syntax.

Answer 2

To click on the element LOAD MORE you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Using CSS_SELECTOR:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[title='Load more items']"))).click()
  

• Using XPATH:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Load more items']"))).click()
  

• Note: You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

Answer 3

this is the XPATH for that button that you want you can right-click on an element to inspect and right-click on it to copy XPath

driver.find_element(By.XPATH, ' //*[@id="block- 
 mainpagecontent"]/div/div/div[2]/div/div/ul/li/a').click()