I've been doing webscrape with BeautifulSoup, Selenium and Scrapy for a few months, mainly for research purposes. After up and downs I always managed to achieve my web-scraping objectives (a lot of them thanks to this site) until I face this site 'https://euclid.eba.europa.eu/register/cir/search' the page uses javascript and needs to be rendered in order to get the results. With selenium, I managed to click on Continue, Select EEA-Brach type and click on search but after getting the page source code of the results I only get script code and not the rendered html/css as usual. Is there a way to scrape this site?
driver.get('https://euclid.eba.europa.eu/register/cir/search')
sleep(1)
movetostring = driver.find_element_by_xpath("/html/body/app-root/app-cir/div[1]/div/app-disclaimer-page/div/div[3]/p")
action.move_to_element(movetostring).perform()
driver.find_element_by_xpath("/html/body/app-root/app-cir/div[1]/div/app-disclaimer-page/div/p-button/button/span[1]").click()
sleep(1)
driver.find_element_by_xpath('//p-card/div/div/div/p-card[1]/div/div/div/div[1]/p-dropdown/div/label').click()
sleep(0.5)
driver.find_element_by_xpath(/html/body/app-root/app-cir/div[1]/div/app-app-entity-search/form/p-card/div/div/div/p-card[1]/div/div/div/div[1]/p-dropdown/div/div[4]/div/ul/li[1]).click()
sleep(1)
driver.find_element_by_xpath('/html/body/app-root/app-cir/div[1]/div/app-app-entity-search/form/p-card/div/div/div/p-card[2]/div/div/div/p-button[1]/button').click()
sleep(2)
html=driver.page_source