0

I need selenium using the firefox driver to scroll down the page to the very bottom, but my code no longer works. Its like the page recognizes I am trying to scroll with selenium and forces the scroll back to the top of the screen...

def scroll_to_bottom(): 
    '''
    Scroll down the page the whole way. This must be done to show all images on page to download.
    '''

    print("[+] Starting scroll down of page. This needs to be performed to view all images.")
    count = 0
    start_time = time.perf_counter()
    try:
        last_height = driver.execute_script("return document.body.scrollHeight")
        while True:
            if count % 10 == 0: 
                elapsed_time = time.perf_counter() - start_time
                elapsed_time_clean = time.strftime("%H:%M:%S", time.gmtime(elapsed_time))
                print("[i] Still scrolling, its been " + str(elapsed_time_clean) + ", please continue to standby... C: " + str(count))
            time.sleep(3)
            driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
            time.sleep(3)

            # Check if hight has changed
            new_height = driver.execute_script("return document.body.scrollHeight")
            vprint("last_height: " + str(last_height) + ", new_height: " + str(new_height))
            if new_height == last_height:
                break
            last_height = new_height
            count += 1

    except Exception as exception:
        print('[!] Exception exception: '+str(exception))
        pass

    print("[i] Scroll down of whole page complete.")
    return

It will go to the page, start to scroll once, then pop up to the top of the screen and no longer scroll. Then my code thinks its at the bottom of the page because the page size did not change. This worked about 3 weeks ago but no longer works. I cant figure out why.

Is there a way to force scrolling? BTw I tried using "DOWN" and "Page DOWN" key presses, that does not work either. Anyone have any ideas?

Dave
  • 727
  • 1
  • 9
  • 20
  • Can you post the URL and the complete code you've tried? – AbiSaran Dec 19 '22 at 02:23
  • https://stackoverflow.com/questions/20986631/how-can-i-scroll-a-web-page-using-selenium-webdriver-in-python - though you have tried this , would selecting an element and sending the command + down keys (mac) or control + down keys (windows) work? Posting the URL would provide us with more context on how to figure out a way to scroll to the bottom. – Andrew Ryan Dec 19 '22 at 03:10
  • There are different kinds of implementation of web pages, not all the pages can be scrolled with regular scroll commands. In order to help you we need to see the specific page you working on and see what will work there using different approaches i.e. by actual debugging – Prophet Dec 19 '22 at 08:03

0 Answers0