I am trying to use selenium to try and scrape this site. (https://suttacentral.net/mn86/en/sujato)
get the element that represents the 'views' button at the upper right corner of the page below the search icon
click on it
When I click on the button and inspect it, this is what I found
<mwc-icon-button class="white-icon toolButtons" id="btnTools" title="View options" slot="actionItems" hidden="" style="display: inherit;"> <!--?lit$530841421$--><svg class="visibility icon" xmlns="http://www.w3.org/2000/svg" height="24" viewBox="0 0 24 24" width="24"><path d="M0 0h24v24H0z" fill="none"></path><path d="M12 4.5C7 4.5 2.73 7.61 1 12c1.73 4.39 6 7.5 11 7.5s9.27-3.11 11-7.5c-1.73-4.39-6-7.5-11-7.5zM12 17c-2.76 0-5-2.24-5-5s2.24-5 5-5 5 2.24 5 5-2.24 5-5 5zm0-8c-1.66 0-3 1.34-3 3s1.34 3 3 3 3-1.34 3-3-1.34-3-3-3z"></path></svg> </mwc-icon-button>
So I thought I could retrieve it with the id 'btnTools'
This is the code I have tried:
from selenium import webdriver
import time
from selenium.webdriver.chrome.options import Options
PATH = "chromedriver"
driver = webdriver.Chrome(PATH)
driver.get("https://suttacentral.net/mn86/en/sujato")
time.sleep(10)
driver.find_element_by_id("btnTools")
and this is the error:
Traceback (most recent call last):
File "/Users/nn/PycharmProjects/1palihelper/muck.py", line 12, in <module>
driver.find_element_by_id("btnTools")
File "/Users/nn/.virtualenvs/1palihelper/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/Users/nn/.virtualenvs/1palihelper/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
'value': value})['value']
File "/Users/nn/.virtualenvs/1palihelper/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Users/nn/.virtualenvs/1palihelper/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="btnTools"]"}
(Session info: chrome=93.0.4577.63)
I have also noticed that when I click 'Developer Tools' and then 'Elements', the code I see there is different to what driver.page_source
returns. Can someone please point in the right direction? driver.page_source
does not return anything in the body.
Thanks Alot