4

I'm trying to download a few hundred excel files from sellercentral.amazon.de. As mentioned in my previous post, manual download is not an option, as I need to make several clicks to get to download popup.

In order to do so, I'm using Python and Selenium.

The Problem

However, the website to scrape doesnt simply consist of buttons and links, but custom tags as well. One of these is the tag 'browse-node-component', each of which represents a product (sub-)category. Finding it is no big deal, but the click on it doesn't get executed. The clicks are needed to navigate through the categories to it's children, until a leaf node is reached. The Icon then changes from an arrow to 'select' (see imgur).

The code I've tried so far is:

elements = driver.find_elements_by_tag_name("browse-node-component")

for element in elements:

    print("starting")

    # Store element name    
    browse_node = element.find_element_by_class_name("browse-node-text")
    browse_node = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "browse-node-text")))

    browse_node_button = element.find_element_by_class_name("a-button-input")
    print ("done")
    browse_node_button.click()

Exemplary "browse-node-component"-tag 

    <span>
        <div class="a-section a-spacing-none browse-node selected-node" ng-class="(nodeCtrl.node.hasChildren ? 'browse-node' : 'leaf-node') + ' ' + (nodeCtrl.isSelected ? 'selected-node' : '')" ng-click="nodeCtrl.node.hasChildren &amp;&amp; nodeCtrl.onBrowseChildren({node:nodeCtrl.node});">

            <div class="browse-node-text ng-binding" style="">Auto &amp; Motorrad</div>

            <div class="node-icon-btn-block">

                <img class="loading-icon ng-hide" src="https://m.media-amazon.com/images/G/01/abis-ui/loading-small._CB192205764_.gif" ng-show="nodeCtrl.isLoading">
                <div class="lock-icon ng-hide" ng-show="nodeCtrl.isGated"></div>
                <div class="a-icon arrow-right browse-node-arrow" ng-show="nodeCtrl.node.hasChildren"></div>

                <div class="select-button-ungated ng-hide" ng-show="!nodeCtrl.isGated &amp;&amp; !nodeCtrl.node.hasChildren" ng-click="nodeCtrl.onSelectNode({node:nodeCtrl.node})">
                    <span class="a-button a-button-base a-button-small select-button"><span class="a-button-inner"><input class="a-button-input" type="submit"><span class="a-button-text" aria-hidden="true">
                       Auswählen
                    </span></span></span>
                </div>

                <div class="gated-button ng-hide" ng-click="nodeCtrl.onRequestApproval({node:nodeCtrl.node})" ng-show="nodeCtrl.isGated &amp;&amp; !nodeCtrl.node.hasChildren">
                    <span class="a-button a-button-primary a-button-small select-button"><span class="a-button-inner"><input class="a-button-input" type="submit"><span class="a-button-text" aria-hidden="true">
                        Freischaltung beantragen
                    </span></span></span>
                </div>

            </div>
        </div>
    <span>
</span></span></browse-node-component>

The Result

The last output is "done", then the script gets terminated and following message appears:

Message: Element 'input class="a-button-input" type="submit"' could not be scrolled into view

But I am unsure to whether I actually have to press that button. Am I pressing the right one? If so, how can I do it without errors?

I want to navigate through those categories, until i reach a child node and then press the select button of it. Yet I cant show the subcategories of toplevel categories with my code. Please give me a hint on which html-element I have to click on.

See here, for graphical information:

Selenium and Python: image taken from https://i.stack.imgur.com/Fx3iw.jpg

Page when instance is opened with selenium: image taken from https://i.stack.imgur.com/Fx3iw.jpg

Image Source : https://i.stack.imgur.com/Fx3iw.jpg

questionasker
  • 2,536
  • 12
  • 55
  • 119
Lino
  • 184
  • 1
  • 1
  • 8
  • I do not know and have not tested this but I highly suspect the solution to your problem is that, as the error message says, you need the element to be visible to be clickable. This is referenced here: https://stackoverflow.com/questions/22588096/selenium-web-driver-cannot-be-scrolled-into-view. To achieve this, apparently you can do this: https://stackoverflow.com/questions/3401343/scroll-element-into-view-with-selenium – ApplePie Apr 27 '19 at 02:01
  • @AppliePie: Thanks, seems like my search got hanged up on clicking custom tags instead of the error message (which doesn't appear on any run). – Lino Apr 27 '19 at 08:52

1 Answers1

0

Try the below.

browse_node_button.location_once_scrolled_into_view
browse_node_button.click()

If the above does not work then you might have to use the JS to click on browse_node_button

driver.execute_script("arguments[0].click();", browse_node_button)
supputuri
  • 13,644
  • 2
  • 21
  • 39
  • And thanks again. I'll have to put your name into the boiler plate ;) Above solution did not work, JS did though. – Lino Apr 27 '19 at 08:51