0

I am trying to scroll down a page using selenium with python and using the execute_script command. However, when I execute my code, I don't scroll down the bottom of my page. Following is my code:

def create_browser(first_page=None):
    print "Starting"
    browser = webdriver.Firefox()
    if first_page:
        browser.get(first_page)
    print "Done."
    return browser

browser = create_browser()
print 'loaded'
browser.get('https://chrome.google.com/webstore/detail/ie-tab/hehijbfgiekmjfkfjpbkbammjbdenadd/reviews')
time.sleep(4)
elem = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, '//a[@ga:type="NextLink"]')))
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
print "Scrolled."

I get the "Scrolled." printed on my console but the page isn't scrolled down. I have read some pages saying that browser.execute_script("window.scrollTo(0, document.body.scrollHeight);") will scroll me to the bottom of my page. However, this doesn't happen in my case. Does this happen to do with the web page using AJAX? Or is it something different? More importantly, how do I scroll down the page? I want to scroll down the page and click on "Next" so that I can see the next reviews and I want to keep doing that until the reviews end. How do I do that?

TheRookierLearner
  • 3,643
  • 8
  • 35
  • 53
  • Other pieces of code i've seen using this technique are waiting a bit before executing it. document.body.scrollHeight might not be ready when you fire the script. Cf : http://stackoverflow.com/questions/21753130/scrolling-down-a-page-with-selenium-webdriver?rq=1 – ddelemeny Mar 09 '14 at 20:34
  • @ddelemeny - I also modified my code (edited above) to wait till the element I want is loaded. However, now I get a timeout exception even though I see that my element is loaded. – TheRookierLearner Mar 09 '14 at 21:09
  • You could try to `alert(window.scrollHeight);` to make sure it has the value you expect it to have (and verify it from the console). – jadkik94 Mar 09 '14 at 21:12
  • @jadkik94 - Nope. It doesn't give me the expected value. When I `alert(window.scrollHeight);` I get the alertbox saying `undefined` – TheRookierLearner Mar 09 '14 at 21:21
  • @TheRookierLearner Oops I meant `window.document.body.scrollHeight`, just realized it when I read it in your comment. If it still doesn't give you a correct value, try either `setTimeout` in the script or `window.onload` or probably a plain `time.sleep` in Python. – jadkik94 Mar 09 '14 at 21:28
  • @jadkik94 - Ya, `alert(window.document.body.scrollHeight);` gives me a value of 5852 but my page still doesn't scroll down. i scroll down to the bottom of the page and I try do `alert(window.document.body.scrollHeight);` in the console and I still get 5852 in the alert box. – TheRookierLearner Mar 09 '14 at 21:36

4 Answers4

1

I had to add a small delay to get it to function. Do not ask me why... 1 ms is not too much to argue about:

<body onload="setTimeout(function(){window.scrollBy(0,document.body.scrollHeight)}, 1);>
nudeln
  • 11
  • 2
0

It seems that on the web page you are visiting, the scrollbar is not actually that of the window you are on. You probably mean to scroll the comment box or the overlay, not the body. Because scrolling the body has no (visible) effect.

So you have to find the element you want to scroll, then scroll that element.

For example, the selector of the comment box (which can be scrolled) is .webstore-te-qc-rb-Lg. Then your code will look something like this:

var a = document.getElementsByClassName("webstore-te-qc-rb-Lg");
// assuming a.length == 1
a[0].scrollTop = a[0].scrollHeight;

That did the job for me, but it all depends what element you want to reach and where the scollbar is. You can find which one it is with the dev tools built into firefox: Inspector > Select element with mouse and find the one which wraps the scrollbar and the content in question.

Hope it helps.

jadkik94
  • 7,000
  • 2
  • 30
  • 39
  • The code here gives me the element's height. But how do I refer to that element in javascript? Giving `window.scrollTo(0,a[0].scrollHeight);` will still refer to the body on which the comment box is overlayed. Doing something like `a[0].scrollTo(0,a[0].scrollHeight);` gives me `TypeError` – TheRookierLearner Mar 09 '14 at 22:39
  • You can only use `scrollTo` on the window (i.e. if you want to scroll the body). If you want to scroll an element, you have to change its `scrollTop` property. You should do `a[0].scrollTop = a[0].scrollHeight;` where `a[0]` is the element you want to scroll. – jadkik94 Mar 10 '14 at 17:37
0

Another way to scroll down is get element with body tag of the page and send :space or :down keys on that element - 

body = driver.find_element(:tag_name, 'body')
body.send_keys :space

or

body.send_keys :down

You can repeat scroll down action till your condition is satisfied.

Alpha
  • 13,320
  • 27
  • 96
  • 163
0

Use this code

JavascriptExecutor jse = (JavascriptExecutor) browser; jse.executeScript("window.scrollBy(0,document.body.scrollHeight || document.documentElement.scrollHeight)", "");

in place of yours: browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Ankit Gupta
  • 776
  • 5
  • 12