How to scroll down on Y-axis in HTML window?

Question

I'm trying to scrape data from a SharePoint site that has a few windows. Apparently, all data must be visible to be able to scrape it (JavaScript). I'm trying to figure out how to scroll down in the window and scrape the data at the same time. The window is identified as:

<div class="md-virtual-repeat-offsetter" role="presentation" style="transform: translateY(0px);">

As I scroll down in this window, the translateY(0px) increases as such: 32px, 64px, 96px, etc., up to 31250px. How can I increase the number of pixels and scrape all HTML elements in this window? I think the code should be something like this.

#first move to the element
wd.execute_script('window.scrollTo(0, arguments[0]);', scroll_size)
#then scroll by x, y values, in this case 10 pixels up
wd.execute_script('window.scrollBy(0, 10000);')

But, when I run that, nothing happens.

I have no clue what you are doing but you may consider emulating PageDown-PageUp button? — İhsan Cemil Çiçek, Jan 15 '19 at 14:57
Using Beautiful Soup and Selenium. Everything works fine, except for this scrolling task. — ASH, Jan 15 '19 at 14:58
Possible duplicate of [Get HTML Source of WebElement in Selenium WebDriver using Python](https://stackoverflow.com/questions/7263824/get-html-source-of-webelement-in-selenium-webdriver-using-python) — Nick Chapman, Jan 15 '19 at 15:11
It seems _incredibly_ unlikely that you would need headless content to be "visible" in order to scrape it — Nick Chapman, Jan 15 '19 at 15:12

How to scroll down on Y-axis in HTML window?

0 Answers0