0

I'm scraping a website that has the following structure:

<div class="cr__stores-list">
  <div class="cr__stores-item">store1</div>
    <a href="/stores/store1/">
    <div class="cr__stores-item-b">
      <div class="cr__container-name">
        <h3 class="cr__container-name-title cr__text--subtitle3 cr__textColor--colorDark300">
          Store Name
        </h3>
      </div>
     </div>
    </div>
  <div class="cr__stores-item">store2</div>
    ...
    ...
  <div class="cr__stores-item">store3</div>
  <div class="cr__stores-item">store4</div>
</div>

Each "div class="cr__stores-item"" inside "div class="cr__stores-list"" has the exact same structure. for each "div class="cr__stores-item"" I'm trying to scrape the url and the store name inside the h3 tag

I have the following code so far:

    wd = webdriver.Chrome('chromedriver', options=options)
    wd.get('stores_url')        
    element = WebDriverWait(wd, 20).until(EC.presence_of_element_located((By.CLASS_NAME, "cr__stores-list")))
    list_stores_tags = element.find_elements_by_class_name("cr__stores-item")
    for store_tag in list_stores_tags:
        # store_tmp = store_tag.find_element_by_class_name("cr__container-name-title cr__text--subtitle3 cr__textColor--colorDark300")
        # store_tmp = store_tag.find_element_by_css_selector('h3.cr__container-name-title cr__text--subtitle3 cr__textColor--colorDark300')
        store= store_tmp.text
        url_tmp = store_tag.find_element_by_tag_name("a")
        url = url_tmp.get_attribute('href')

The 2 commented lines are my attempts at getting the store name. With this code I'm successfully getting all urls of each store without problem. However I'm unable to get the store names. Neither find_element_by_class_name nor find_element_by_css_selector seem to work as in both cases I get an exception:

Message: no such element: Unable to locate element: {"method":"css selector","selector":".cr__container-name-title cr__text--subtitle3 cr__textColor--colorDark300"} (Session info: headless chrome=87.0.4280.88)

or

Message: no such element: Unable to locate element: {"method":"css selector","selector":"h3.cr__container-name-title cr__text--subtitle3 cr__textColor--colorDark300"} (Session info: headless chrome=87.0.4280.88)

What am I doing wrong? Any suggestions would be welcome. Thanks!

Alain
  • 339
  • 3
  • 19
  • You're using `find_element_by_css_selector` incorrectly. Try `store_tag.find_element_by_css_selector('h3.cr__container-name-title.cr__text--subtitle.cr__textColor--colorDark300')`. Note the dots instead of spaces. –  Dec 21 '20 at 15:35
  • Ah! That's it! Thank you! I had no idea that was the way to write tags with spaces ;) – Alain Dec 21 '20 at 15:39
  • That's how you use CSS selectors to choose elements with multiple classes. See https://stackoverflow.com/questions/2554839/select-element-based-on-multiple-classes. –  Dec 21 '20 at 15:44

1 Answers1

0

You're using find_element_by_css_selector incorrectly. Try

store_tag.find_element_by_css_selector('h3.cr__container-name-title.cr__text--subtitle.cr__textColor--colorDark300')

Note the dots instead of spaces.