-2

I am trying to get the HTML from a webpage that is modified by javascript after/while loading. I have followed the instructions in this tutorial. and I'm using command like this in my Python code after initially loading the page

html = browser.execute_script("return document.body.innerHTML")

While this seems to get all of the HTML Selenium elements on other pages I've tried this on, it does not seem to work on this page. If you open that page and inspect the HTML you can see all of the HTML has loaded. I want to capture the elements below, but can't. By the way WebDriverWait doesn't seem to work either...

<b>
  <a href="contractor_list.asp?alpha=A">A&nbsp;</a>
  <a href="contractor_list.asp?alpha=B">B&nbsp;</a>
  <a href="contractor_list.asp?alpha=C">C&nbsp;</a>
  <a href="contractor_list.asp?alpha=D">D&nbsp;</a>
  ...

How can I get Selenium to expose those elements to me so I can access them? Or, should I be using a different tool for this?

Jed
  • 1,823
  • 4
  • 20
  • 52
  • Thanks for the link to the other SO question related to this. I tried to find a related question prior to posting, but at that time didn't know I should be using "iframe" as one of my search keywords. – Jed Sep 30 '17 at 02:45

1 Answers1

0

Those elements are contained within an iframe element, which is why you're unable to interact with them. Try running the script on the iframe's source if possible.

B. Fleming
  • 7,170
  • 1
  • 18
  • 36
  • How do I explicitly run the script on that element to expose the rest of the elements? – Jed Sep 30 '17 at 00:03
  • I don't think that's possible, actually. I recall using Selenium once in the past and ran into this exact same issue a few times. Unfortunately you'll just have to inspect the element manually, find the iframe, grab its source, and then visit that specific page and run your Selenium script from there. Perhaps it's possible to automate this process? Unfortunately it's been a few years, so I don't really have any advice for setting up the appropriate automation. – B. Fleming Sep 30 '17 at 00:09