I apologise in advance for the (probably) very basic question. I spent a lot of time searching forums but my knowledge is too poor to make sense of the results.
I just need to get the HTML after the page has finished loading as almost all of the content is stored in div id="root">/div> but at the moment i just get that one line and nothing inside it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
browser = webdriver.Chrome() #replace with .Firefox(), or with the browser of your choice
url = "https://beta.footballindex.co.uk/top-200"
browser.get(url) #navigate to the page
innerHTML = browser.execute_script("return document.body.innerHTML") #returns the inner HTML as a string
print(innerHTML)
Returns:
<div id="root"></div>
<script src="https://static.footballindex.co.uk/bundle_1537553245755.js"></script>
And this matches the innerHTML when you 'view page source'. But if i inspect element in my browser you are able to expand div id="root">/div> to see all the content inside and then I can manually copy all the HTML.
How do i get this automatically?
Many thanks in advance.
