Trying to click amazon Best Sellers Rank (Python)

Question

Hello i'm trying to click those links but when I try to click with

driver.find_element_by_xpath('//*[@id="productDetails_detailBullets_sections1"]/tbody/tr[6]/td/span/span[2]/a').click()

its work's but problem is every items has different path and its changing and it does not work for some items

URL: https://www.amazon.com/MICHELANGELO-Piece-Rainbow-Kitchen-Knife/dp/B074T6C4YS/ref=zg_bs_289857_1?_encoding=UTF8&psc=1&refRID=K5GAX1GF2SDZMN3NS403>

Answer 1

The Amazon webpage has 3 entries for Best Sellers Rank. An effective approach would be to collect the href of all the three(3) Best Sellers, store them in a list and open in a seperate tab to scrape. To construct the list you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR:

  driver.get('https://www.amazon.com/dp/B074T6C4YS')
  print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "table#productDetails_detailBullets_sections1 td>span>span a")))])
  

• Using CSS_SELECTOR in a single line:

  driver.get('https://www.amazon.com/dp/B074T6C4YS')
  print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@id='productDetails_detailBullets_sections1']//td/span/span//a")))])
  

• Console Output:

  ['https://www.amazon.com/gp/bestsellers/kitchen/ref=pd_zg_ts_kitchen', 'https://www.amazon.com/gp/bestsellers/kitchen/289857/ref=pd_zg_hrsr_kitchen', 'https://www.amazon.com/gp/bestsellers/kitchen/289862/ref=pd_zg_hrsr_kitchen']
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

Answer 2

This is easy one, even you did not specify which link do you want, only from the table all different links that will transfer you to table.

You need to use customized xpath like e.g.

//*[@id="productDetails_detailBullets_sections1"]/tbody/tr[6]/td/span/span['+i+']/a'

Where i will be your iterator in for loop. To get valu of i use something like

driver.find_elements_by_xpath('//*[@id="productDetails_detailBullets_sections1"]/tbody/tr[6]/td/span/span').size();