Web Scraping tradingview chart with python selenium

Question

I am trying to scrape data from tradingviews charts. This is the code that I am using and so far is working

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time

MAMAarray = []
FAMAarray = []

# example option: add 'incognito' command line arg to options
option = webdriver.ChromeOptions()
option.add_argument("--incognito")

# create new instance of chrome in incognito mode
browser = webdriver.Chrome(chrome_options=option)

# go to website of interest
browser.get("https://www.tradingview.com/chart/vKzVQllW/#")

# wait up to 10 seconds for page to load
timeout = 10
try:
    WebDriverWait(browser, timeout).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[1]")))
except TimeoutException:
    print("Timed out waiting for page to load")
    browser.quit()


time.sleep(5)

# get MAMA and FAMA values for BTCUSDT on binance
MAMA_element = browser.find_element_by_xpath("/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[3]")
FAMA_element = browser.find_element_by_xpath("/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[4]")

MAMAarray.append(float(MAMA_element.text))
FAMAarray.append(float(FAMA_element.text))

print(MAMAarray)
print(FAMAarray)

while True:
    MAMA_element = browser.find_element_by_xpath(
        "/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[3]")
    FAMA_element = browser.find_element_by_xpath(
        "/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[4]")
    if (float(MAMA_element.text)) != MAMAarray[-1]:
        MAMAarray.append(float(MAMA_element.text))
    if (float(FAMA_element.text)) != FAMAarray[-1]:
        FAMAarray.append(float(FAMA_element.text))

    print(MAMAarray)
    print(FAMAarray)

Here is the output: so you can see that the numbers append but ONLY when I go into the chart manually and move my cursor over new candles(bars)

[7415.969]
[7417.39]
[7415.969, 7428.644]
[7417.39, 7435.585]
[7415.969, 7428.644, 7430.56]
[7417.39, 7435.585, 7431.722]
[7415.969, 7428.644, 7430.56, 7415.496]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39, 7424.887]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969, 7439.161]
[7417.39, 7435.585, 7431.722, 7417.39, 7424.887, 7424.409]

How do I go about automating this so that it automatically gets the newest value instead of me having to cursor over the new bar for it to get the new value

EDIT:

This is the value Im looking for Like I said I can get it to output through the script, but I cant get the value to update when a new candle is made

this

Answer 1

If I correct understood you, you need a hoverover an element. To be able to do it, you can try this:

hover = ActionChains(webDriver).move_to_element(element_to_hover_over)
hover.perform()

Also you can have a look to Documentation

EDIT:

In your case I would use this two selectors to get the information you need:

//span[@class='dl-ask']
//span[@class='dl-bid']

They are changing dynamically without hover need.

Output will be like this:

7420.06x2 // you have to split this string and extract 7420.06
7424.00×0.007977999746799469 // and here 7424.00

Since you want to get dynamically from elements you provided, there is a solution for it. I wouldn't use it for every problem, because it is not a "clean" one. But for our specific case I think it is a best one.

I have found that if you hover to particular candle in the graphic, every minute when graphic will be updated, all candles will be shifted to the left. Also if you hover on (x,y) position which is (y) position for one of the candles, you get the value of the candle. And that is exactly what you need. My propose is to hover to the candle you need and the values from your elements will be updated since the values this candle be updated(every 1 minute). If you want the information with minimum delay, you have to hover on the right candle.

Use this to move to the mouse to your candle:

move_to_element_with_offset(to_element, xoffset, yoffset)

Move the mouse by an offset of the specified element. Offsets are relative to the top-left corner of the element. Args:

• to_element: The WebElement to move to.
• xoffset: X offset to move to.
• yoffset: Y offset to move to.

Also you may want to zoom in the graphic. You can try the suggestions here, or:

webdriver.execute_script("window.scrollBy(0, 150);") // 0 - xPos, 150 - yPos (values are in px).