How to scroll down a web page to verify a specific element?

Question

I'm learning how to use selenium and I'm stuck on figuring out how to scroll down in a website to verify an element exists.

I tried using the methods that was found in this question Scrolling to element using webdriver?

but selenium won't scroll down the page. Instead it'll give me an error

"selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: element"

Heres the codes I am using

moveToElement:

element = driver.find_element_by_xpath('xpath')

actions = ActionChains(driver)

actions.move_to_element(element).perform()

Scrolling into View

element = driver.find_element_by_xpath('xpath')

driver.execute_script("arguments[1].scrollIntoView();", element)

The whole code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Firefox()
driver.get("https://www.linkedin.com/")

element = 
driver.find_element_by_xpath('/html/body/div/main/div/div[1]/div/h1/img')

element = driver.find_element_by_xpath('//*[@id="login-email"]')
element.send_keys('')

element = driver.find_element_by_xpath('//*[@id="login-password"]')
element.send_keys('')

element = driver.find_element_by_xpath('//*[@id="login-submit"]')
element.click();

element = driver.find_element_by_xpath('')

actions = ActionChains(driver)

actions.move_to_element(element).perform()

are you talking about the link I'm trying to use to find the element? — kav, Jan 04 '18 at 05:34
@use724900 should I be using `find_element_by_id` instead? I'm use to finding element through xpath — kav, Jan 04 '18 at 05:50
@Murthi I tried `time.sleep(100)` still it won't scroll down to verify the element — kav, Jan 04 '18 at 05:53
I am still not able to get you.you are tying to move at which element? — Nimish Bansal, Jan 04 '18 at 06:02
@NimishBansal I am trying to verify an element exists inside a webpage. But you will need to scroll down the webpage to verify the element exists. So I'm trying to scroll down the page to verify the element exists. — kav, Jan 04 '18 at 06:11

Amol Chavan · Answer 1 · 2018-01-04T07:08:55.590

There are two aspects to your is problem

Whether the element exist?
Whether the element displayed on the page?

Whether the element exist?

It may happen that the element exists on the page[i.e. it is part of the DOM] but it is not available right away for further selenium action becuase it is not visible[hidden], it's only gets visible after some actions or it may get displayed on scroll down the page.

Your code is throwing an exception here-

element = driver.find_element_by_xpath('xpath')

as WebDiver not able to find the element using mentioned xpath. Once you fix this, you can moe forward the next part.

Whether the element displayed on the page?

Once you fix the above issue, you should check whether the element is being displayed or not. If it's not displayed and avialble on the scroll then you can use code like

if !element.is_displayed():
  driver.execute_script("arguments[1].scrollIntoView();", element)

Perfer using Action Class for very specific mouse action.

Update:-

If you are application using lazy loading and the element you are trying to find is available on scroll the you can try something like this -

You have to import exception like -

from selenium.common.exceptions import NoSuchElementException

and the create new recursion function which would scroll if element not found, something like this -

    def search_element():
      try:
        elem = driver.find_element_by_xpath("your")
        return elem
    except NosSuchElementException:
        driver.execute_script("window.scrollTo(0,Math.max(document.documentElement.scrollHeight,document.body.scrollHeight,document.documentElement.clientHeight));")
        search_element()

I am not sure that having recurion for finding element here is good idea, also I have naver worked on the python so you need to lookout for the syntax

Hi @Amol, thanks for your answer! Is there a way for selenium to keep scrolling down until the element is found? — kav, Jan 04 '18 at 06:51
@Amil Chavan NoSuchElementException: Message: no such element: Unable to locate element — rsc05, Aug 21 '22 at 11:56

Nimish Bansal · Answer 2 · 2018-01-04T07:14:54.560

0

Amm may be this might help.Just send page down key and if you are sure that element exists definitely then this would work

from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
import time
while True:
    ActionChains(driver).send_keys(Keys.PAGE_DOWN).perform()
    time.sleep(2) #2 seconds
    try:
        element = driver.find_element_by_xpath("your_element_xpath")
        break
    except:
        continue

edited Jan 04 '18 at 07:14

answered Jan 04 '18 at 06:45

Nimish Bansal

1,719
4
20
37

Hi @Nimish Bansal Thanks for you answer! Its scrolling down! But it is scrolling down too fast to the point it won't see the xpath. Is there a way to control the speed of the scroll? – kav Jan 04 '18 at 07:00

How to scroll down a web page to verify a specific element?

2 Answers2