2

I'm a bit new when it comes to webscraping and browser automation, so I'd appreciate your help!

My goal using Python's Selenium package is to:

  1. Go to http://www.mcmaster.com/#orders/
  2. Log in using the username & password on the side
  3. Fill out my order in the Quick Order function

The problem is that I identified the xpath for each login field for #2 with this code:

def orderMcMasterParts():
    # Open each of the websites required
    mcmBrowser = webdriver.Firefox()
    mcmBrowser.get('http://www.mcmaster.com/#orders/')

    # Log in with my credentials
    usernameElem = mcmBrowser.find_element_by_xpath("//input[@class='LoginWebPartLayout_InpBx' and @type='text']")
    usernameElem.send_keys(mcmUsername)

    passwordElem = mcmBrowser.find_element_by_xpath("//input[@class='LoginWebPartLayout_InpBx' and @type='password')
    passwordElem.send_keys(mcmPassword)
    passwordElem.submit()

But I continue to get a NoSuchElementException. When I download the HTML on that page with the requests module, I see that none of these fields appear in the HTML code. They appear to exist as Javascript snippets.

My question is how do I interact and type into these fields?

alexis
  • 48,685
  • 16
  • 101
  • 161
Mach3Maelstrom
  • 21
  • 1
  • Some more information that may help: When I right-click the Username field in Firefox and choose "Inspect Element", I get HTML with class "csstransitions gr__mcmaster_com" But if I right-click the page and choose "View Page Source", I get HTML that starts with a header with class="". – Mach3Maelstrom Dec 04 '15 at 20:15
  • 1
    Is the javascript code automatically executed? If not, then you should trigger it execution, wait a few secs, and then try to get the input. – Ciro Pedrini Dec 04 '15 at 20:44
  • Check [this answer](http://stackoverflow.com/a/26567563/2697279), it shows you how to wait until some element is usable. – jehna1 Dec 04 '15 at 20:45

1 Answers1

0

The issue occurs if you don't have element on the page. One of the reason may be that you need to wait till element will be loaded. You have at least 2 options: explicit WebDriver wait and implicit WebDriver wait.

If we are talking about concrete example: http://www.mcmaster.com/#orders/

2 things came to my mind (after quick debugging):

  • this "Log In" form is inside iframe: before accessing element inside iframe you need to switch to iframe through Selenium
In [17]: driver.switch_to.frame?
Type:        instancemethod
File:        /usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/switch_to.py
Definition:  driver.switch_to.frame(self, frame_reference)
Docstring:
Switches focus to the specified frame, by index, name, or webelement.

:Args:
 - frame_reference: The name of the window to switch to, an integer representing the index,
                    or a webelement that is an (i)frame to switch to.

:Usage:
    driver.switch_to.frame('frame_name')
    driver.switch_to.frame(1)
    driver.switch_to.frame(driver.find_elements_by_tag_name("iframe")[0])
  • make sure that you selected right web element before typing something; web element should be visible
drets
  • 2,583
  • 2
  • 24
  • 38