1

Let's say I want to scroll a scrollbar in some element. As an example, let's take the link "https://www.w3schools.com/howto/howto_css_table_responsive.asp". If I run:

element = driver.find_element_by_xpath("//table")

I get the table element. But now, I want to scroll the scrollbar horizontally, that is controling the view of this table. How can I do this? I already tried something like:

driver.execute_script("arguments[0].scrollLeft = 200;",element)

But I wasn't successfull. I also tried sending keys, but it didn't work too.

Rodolfo
  • 96
  • 7
  • First of all, what are you trying to *accomplish* - I.E. WHY are you trying to scroll to the right, before we get to HOW to scroll to the right. Using Selenium you rarely need to scroll within a table to continue targeting elements within that table unless it is scroll to load - which is almost always [from what I have seen] triggered by scrolling down, not right. – Cfomodz Jun 30 '21 at 16:43
  • I really want to reveal the elements from scrolling down (in another website). But this example (the link) was the simplest I was able to find to put it here. If I can scroll right in this table, I think I can scroll down in others ;) – Rodolfo Jun 30 '21 at 16:46
  • Are you wanting to scroll down to be able to have it js load more elements or just to put them into the user's view? Again, what are you actually trying to accomplish? WHY do you want to scroll down to reveal those elements? – Cfomodz Jun 30 '21 at 16:58
  • Seems like a duplicate of [How can I scroll a web page using selenium webdriver in python?](https://stackoverflow.com/questions/20986631/how-can-i-scroll-a-web-page-using-selenium-webdriver-in-python) – Cfomodz Jun 30 '21 at 17:01
  • The website I want to navigate is, for example, this: "https://www.morningstar.com/stocks/xnas/tsla/financials". You then open the link with name "Income Statement" and you get to a table of values. Some values are hidden and only appear if I scroll the scrollbar. How can I get those elements (values)? – Rodolfo Jun 30 '21 at 17:24
  • When you say some values are hidden do you mean they are not visible to you as a human or that they are not populated in the DOM? Having just opened your link and looked at the table in question I would bet all the bitcoin I own that you can just target the elements that you want without scrolling in any direction. – Cfomodz Jun 30 '21 at 17:37

1 Answers1

0

You do not need to actually be able to see elements for Selenium to target them.

Just proceed to target the elements that are 'off screen' as you would if they were on screen.

Selenium is targeting the DOM - which as a loose analogy which I will probably be roasted for in the comments is like a screen with infinite length and width - so no scrolling necessary - it (selenium) can already "see" everything 'on' that 'infinite screen'

If you are having trouble selecting specific elements from within the table in question (if you need a specific code snippet) please feel free to comment - I am rather familiar with Selenium.

EDIT:

In your specific case, since they are being loaded dynamically you are 100% correct that you will need to scroll to get the data. Let's take care of vertical scrolling by hitting the More Financial Data button at the bottom that expands the list.

Now for scrolling to the right - since we can just scroll all the way to the right and have all data that we need to access visible (assuming you don't have a Premium account).

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Firefox()
url = "https://www.morningstar.com/stocks/xnas/tsla/financials"
driver.get(url)
driver.maximize_window()
# Click on Income Statement
xpath = '//*[@id="__layout"]/div/div[2]/div[3]/main/div[2]/div/div/div[1]/sal-components/section/div/div/div/div/div[2]/div/div/div[2]/div[2]/div/div[2]/div/div[2]/div[2]/div[1]/div[1]/a'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath))).click()
# click on More Financials Detail Data
xpath = '//*[@id="__layout"]/div/div[2]/div[3]/main/div[2]/div/div/div[1]/sal-components/section/div/div/div/div/div[2]/div/div[2]/div/div[2]/div/div[3]/div[2]/div/a'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath))).click()

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

xpath='/html/body/div[2]/div/div/div[2]/div[3]/main/div[2]/div/div/div[1]/sal-components/section/div/div/div/div/div[2]/div/div[2]/div/div[2]/div/div[2]/div[4]/div/div[3]/div[2]/div[2]'
horizontal_bar_width = driver.find_element_by_xpath(xpath).rect['width']
slider = driver.find_element_by_xpath(xpath)
ActionChains(driver).click_and_hold(slider).move_by_offset(horizontal_bar_width/2, 0).release().perform()
Cfomodz
  • 532
  • 4
  • 17
  • Ok, thank you. I will try again tomorrow as if the elements are not hidden. I will report back – Rodolfo Jun 30 '21 at 17:53
  • Ok, so.. I tried. And I can't select the elements not visible to me on the table. If I go to inspector, the elements don't appear there until I scroll to the right (or down). Without any scrolling, can you retrieve the values of Total Revenue (or elements holding the values) from 2019 ? – Rodolfo Jun 30 '21 at 18:35
  • Okay, take a look at that and let me know what you think. I have to run but I can check back tomorrow with that right arrow key code snippet if you need it – Cfomodz Jun 30 '21 at 23:00
  • You can open the elements below clicking in "More Financial Data". However, if you could help me scroll right or retrieve the values I would much appreciate it. Btw, don't use such xpath searches. You can use something like this: driver.find_element_by_xpath("//*[normalize-space(text())='More Financials Detail Data']").click() – Rodolfo Jul 01 '21 at 10:04
  • @Rodolfo - Fixed, feel free to fix my xpaths or use other selectors to adapt my code to work with all of the different pages you may want to use it on :) – Cfomodz Jul 01 '21 at 18:36
  • Btw, I was able to find the right element and send_keys to it for scrolling. – Rodolfo Jul 04 '21 at 08:59