Click a partial div link using python selenium

Question

I want to scrape some information on this website: https://www.caixabank.es/particular/prestamos-personales/prestamos.html

I coded to get to the simulation page, but a link crashes me.

dynamic link

I can't click on this link.

My landing page is the one

landing page

My code:

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--start-maximized')
chrome_options.add_argument('window-size=5000x2500')
webdriver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
 
now = DT.now()
periode = str(now.day) + '_' + str(now.month) + '_' + str(now.year)
#url = "https://www.caixabank.es/particular/prestamos-personales/prestamos.html"
url = "https://loc2.caixabank.es/GPeticiones?CANAL=I&DEMO=0&PN=LGN&PE=116&ENTORNO=1&sinCampanya=S&INICIAL_PN=PPP&INICIAL_PE=832&E_COD_LANDING=WGN00645&E_PNMENU=POS&E_PEMENU=6&TEXTO_OPE=prestamoestrellaautoes.txt&IDIOMA=02"
webdriver.get(url)
link = webdriver.find_element_by_partial_link_text('Demostración')
actions = ActionChains(webdriver).click(link)
actions.perform()
time.sleep(5)
webdriver.save_screenshot('sreenshot.png')
webdriver.implicitly_wait(30)
webdriver.switch_to.frame(webdriver.find_element_by_css_selector('#FramesetInferiorInterno > frame:nth-child(2)'))
webdriver.find_element_by_xpath("//a[contains(., 'Aceptar y continuar')]")
webdriver.save_screenshot('sreenshot1.png')

The error I got is:

/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:8: DeprecationWarning: use options instead of chrome_options
  

---------------------------------------------------------------------------

NoSuchElementException                    Traceback (most recent call last)

<ipython-input-81-858e6e0c3de3> in <module>()
     29 webdriver.save_screenshot('sreenshot.png')
     30 webdriver.implicitly_wait(30)
---> 31 webdriver.switch_to.frame(webdriver.find_element_by_css_selector('#FramesetInferiorInterno > frame:nth-child(2)'))
     32 #webdriver.switch_to.frame('Cos')
     33 #link = webdriver.find_element_by_partial_link_text('Aceptar y continuar')

/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py in find_element_by_css_selector(self, css_selector)
    596             element = driver.find_element_by_css_selector('#foo')
    597         """
--> 598         return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
    599 
    600     def find_elements_by_css_selector(self, css_selector):

/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py in find_element(self, by, value)
    976         return self.execute(Command.FIND_ELEMENT, {
    977             'using': by,
--> 978             'value': value})['value']
    979 
    980     def find_elements(self, by=By.ID, value=None):

/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
    319         response = self.command_executor.execute(driver_command, params)
    320         if response:
--> 321             self.error_handler.check_response(response)
    322             response['value'] = self._unwrap_value(
    323                 response.get('value', None))

/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    240                 alert_text = value['alert'].get('text')
    241             raise exception_class(message, screen, stacktrace, alert_text)
--> 242         raise exception_class(message, screen, stacktrace)
    243 
    244     def _value_or_default(self, obj, key, default):

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#FramesetInferiorInterno > frame:nth-child(2)"}
  (Session info: headless chrome=72.0.3626.121)
  (Driver info: chromedriver=72.0.3626.121,platform=Linux 4.14.79+ x86_64)

Don't change the question now. The question has been answered. See my comment below. — JeffC, Mar 12 '19 at 14:06
My question is how to click on the link. I still can't get it. — Tiffany Zucman, Mar 12 '19 at 14:09
Because you no longer get the error that you posted... which is currently still in your question. You need to clean up the code in your question. Remove all the commented out code and post the current code you are using along with the current error message. — JeffC, Mar 12 '19 at 14:11
You need to specify the frame as a WebElement. I think the frame you want can be found this way: `webdriver.switch_to.frame(webdriver.find_element_by_css_selector('#FramesetInferiorInterno > frame:nth-child(2)'))` — C. Peck, Mar 12 '19 at 14:28
I’m pretty sure that xpath isn’t working because the page has layered frames, and the frame you want to use is inside another frame’s #document. I can’t find a way to get inside the #document and I think it might not be possible. I’m going to raise a question specific to the layered frame issue to see if anyone can say definitively if and how you can grab such an element with selenium. — C. Peck, Mar 12 '19 at 19:06
Thanks a lot for your help. For the time you have spent on my question, I will validate your answer. But please, if you have a solution let me go. I'm a beginner in scraping and it's a little complicated for me. — Tiffany Zucman, Mar 12 '19 at 19:12
That website is very messy and hard to navigate with selenium. Don’t worry that you couldn’t get it on your first try— it might even be impossible! — C. Peck, Mar 12 '19 at 19:16
@TiffanyZucman Seems it would be impossible to grab that element with Selenium. I asked a more generic version of your issue [here](https://stackoverflow.com/questions/55130153/if-possible-how-can-i-use-selenium-to-identify-elements-that-reside-in-nested-f) — C. Peck, Mar 12 '19 at 22:47
@C.Peck Thank you so much for your help. I asked another question here https://stackoverflow.com/questions/55099298/unable-to-locate-element-selenium-webdriver/55125742#55125742. If you manage to move the slider of the site to change the amount and duration I would be grateful. — Tiffany Zucman, Mar 13 '19 at 08:00

score 2 · Accepted Answer · edited Mar 12 '19 at 11:32

2

I think you probably just need parentheses around your xpath instead of brackets:

webdriver.find_element_by_xpath("//a[contains(., 'Aceptar y continuar')]")

Does that one work?

edited Mar 12 '19 at 11:32

Sers

12,047
2
12
31

answered Mar 12 '19 at 11:29

C. Peck

3,641
3
19
36

No sadly NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[contains(., 'Aceptar y continuar')]"} – Tiffany Zucman Mar 12 '19 at 12:58
So that did fix your error, it just gave you a new one. The link was not found because it's in an `IFRAME` so you will have to switch to it before you can locator your element. – JeffC Mar 12 '19 at 13:27
I think the new error comes from `(EC.element_to_be_clickable`. I think you need to define `EC = ExpectedConditions` if you want to use it that way. However, I think if you just add the following to your imports you might be good to go: `from selenium.webdriver.support import expected_conditions as EC` – C. Peck Mar 12 '19 at 13:37
@JeffC I tried to switch to Iframe. But not worked. Can you please post an example ? – Tiffany Zucman Mar 12 '19 at 13:47
There are plenty of examples on the web, I'm sure if you searched you can find one. – JeffC Mar 12 '19 at 13:49
@JeffC I've been looking everywhere. I've trying everything, but really everything. I updated my question. – Tiffany Zucman Mar 12 '19 at 13:50
@TiffanyZucman You need to specify the frame as a WebElement. I think the frame you want can be found this way: `webdriver.switch_to.frame(webdriver.find_element_by_css_selector('#FramesetInferiorInterno > frame:nth-child(2)'))` – C. Peck Mar 12 '19 at 14:33
@C.Peck I tried NoSuchElementException: Message: no such element: Unable to locate element – Tiffany Zucman Mar 12 '19 at 14:34
Can you update the code in your question with the latest version which causes this error? That xpath works for me. – C. Peck Mar 12 '19 at 14:49
@C.Peck Updated :) – Tiffany Zucman Mar 12 '19 at 15:43

Click a partial div link using python selenium

1 Answers1

Linked