I am trying to scrape highcharts from two different websites, I came across this execute_script answer in this stackoverflow question : How to scrape charts from a website with python?
It helped me scrape from the first website but when i use it on the second website it returns the following error:
line 27, in <module>
temp = driver.execute_script('return window.Highcharts.charts[0]'
selenium.common.exceptions.JavascriptException: Message: javascript error: Cannot read
property '0' of undefined
The website is : http://lumierecapital.com/#
You're supposed to click on the performance button on the left to get the highchart.
Goal: i just want to scrape the Date and NAV per unit values from it
Like the last website, this code should've printed out a dict with X and Y as keys and the date and data as values but it doesn't work for this one.
Here's the python code:
from bs4 import BeautifulSoup
import requests
from selenium.webdriver.chrome.options import Options
from shutil import which
from selenium import webdriver
import time
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_path = which("chromedriver")
driver = webdriver.Chrome(executable_path=chrome_path, options=chrome_options)
driver.set_window_size(1366, 768)
driver.get("http://lumierecapital.com/#")
performance_button = driver.find_element_by_xpath("//a[@page='performance']")
performance_button.click()
time.sleep(7)
temp = driver.execute_script('return window.Highcharts.charts[0]'
'.series[0].options.data')
for item in temp:
print(item)