1

I am trying to scrape the target website for product_links. The program should open the required URL in the browser and scrape all the links with a particular class name. But for some reason, I am getting a NoSuchElementException for this piece of code

links = driver.find_elements_by_class_name("styles__StyledTitleLink-mkgs8k-5")

for link in links:
    self.driver.implicitly_wait(15)
    product_links.append(link.find_element_by_css_selector('a').get_attribute('href'))

I tried printing out the text in each link with link.text in the for loop. The code is actually selecting the required elements. But for some reason is not able to extract the href URL from each link. I am not sure what I am doing wrong.

This is the entire error message

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"a"} (Session info: chrome=83.0.4103.106)

Sashaank
  • 880
  • 2
  • 20
  • 54

2 Answers2

2

Error seems there is no css element with 'a' so you need to try with other locators to identify elements. try with xpath=//a[contains(text(),'text of that element')]

Justin Lambert
  • 940
  • 1
  • 7
  • 13
1

You are looking for a class name generated by a builder, check the random string at the end of the class name, these classes won't be found in every web.

if you want to scrape them, find a different generic class or find all classes with a subset string "StyledTitleLink"

Here's how to do it with JQuery

You should try and find a different solution to your problem

Natanhel Poliszuk
  • 100
  • 8