1

I'm scraping a page that I cannot share because it's private and needs access. However, I have a box in the page which is a div container that I need to scroll down in order to get all HTML loaded. I tried many ways but scrolling down is still fast and is not waiting for all elements to load. So I need a way to slow down scrolling in a div container

My code so far:

dialog = driver.find_element_by_xpath("//div[@id='search-results-container']")
driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", dialog)

This is working and scrolling down to the bottom of the page but it's fast. I need to slow it down

Any idea?

JM17
  • 149
  • 1
  • 11

2 Answers2

0

I suggest to use WebDriverWait selenuim function to wait for your element to load check this answer https://stackoverflow.com/a/26567563/18317391

Mouad Slimane
  • 913
  • 3
  • 12
0

You can try something like this

while True:
    driver.execute_script(
        """
        container = document.querySelector("<div container css selector>")
        container.scrollBy(0,10)
        """
        )
    # sleep time if it takes long to load(probably won't be needed)
    # stop condition

This way the div container will scroll slowly (10px) until the stop condition is met

Atehe
  • 63
  • 4