1

I am trying to scrape the List of Trades from this TradingView chart(https://www.tradingview.com/chart/lUsimB6z/) but have no idea on where to start. I tried Selenium first but the table loads when you scroll down, which caused a problem for me. I am new to Selenium and Python in general. Any help will be appreciated. Thank you.

Brian Zheng
  • 439
  • 5
  • 18

2 Answers2

2

So, the starter code is as follows:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

The first step would be to create a webdriver object.

chrome_options = Options()
# Stops the UI interface (chrome browser) from popping up
# chrome_options.add_argument("--headless") 
driver = webdriver.Chrome(executable_path='/path/to/chrome_driver', options=chrome_options)

Spin up the browser with the URL

page_url = "https://www.tradingview.com/chart/lUsimB6z/"
driver.get(page_url)

This driver element contains methods that you can use to fetch elements from the html. The complete page source can be obtained by using driver.page_source. You can switch over to BeautifulSoup if needed, by simply using soup = BeautifulSoup(driver.page_source, "html.parser")

Some of the methods you can use with the driver object are:

driver.find_element_by_tag_name(tag_name)
driver.find_element(s)_by_class_name(class_name)
driver.save_screenshot()
etc...

This link explains how you can simulate scrolling on the browser using the driver object to fetch the table data.

Finally,

driver.quit()

You'll need to have chromedriver be located at '/path/to/chrome_driver' for selenium to work.

tidakdiinginkan
  • 918
  • 9
  • 18
0

Just in case anyone is interested in this if you want to scrape very small timeframes (1s, 5s). The method described above poses an issue because the website updates every 10 seconds. The way I found is described below:

  1. use chrome or brave
  2. add the autoview extension
  3. it will add a new tab in the Strategy tester, named 'Export'
  4. use selenium to click on that button to save the strategy results to clipboard and then paste them into a dataframe
  5. export to csv

here's a sample of the code:

#click the Strategy tester tab so that the button is visible
c = driver.find_element_by_xpath('//*[@id="footer-chart-panel"]/div[1]/div[1]/div[4]/div/span')
driver.execute_script("$(arguments[0]).click();", c)

#click the Export button
z = driver.find_element_by_xpath('//li[@class="autoview-backtesting-export"]')
driver.execute_script("$(arguments[0]).click();", z)

If any of this fails, try and add a loop (0-5) and put the action within the loop. The rest is pandas-related.

#copy the clipboard into dataframe    
a = pd.read_clipboard(sep='\t', usecols=[])
Silviu
  • 135
  • 1
  • 9