0

I want to scrape some data from a genealogy website, I need to log in but I can't click with Selenium the Submit button, it may be because I need to accept the cookies but I can't click it as well.

Here is the code:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

PATH = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(PATH)
driver.implicitly_wait(10)

driver.get("https://en.geneanet.org/connexion/")
driver.find_element_by_xpath('//*[@id="tarteaucitronPersonalize2"]').click()
driver.find_element_by_id("_username").send_keys('user')
driver.find_element_by_id ("_password").send_keys("pwd")
driver.find_element_by_id("_submit").click()

The button looks like this:

enter image description here

This is the HTML corresponding to the button "Submit": Submit

The cookies popup and HTML looks like this:

enter image description here

I get the following error messages:

(vscrap) admin@Admins-MacBook-Pro V2 % /Users/admin/Documents/Coding/Python/Scrapping/vscrap/bin/python /Users/admin/Documents/Coding/Python/Scrapping/UpWork/V2/Selenium_login-immo.py
Traceback (most recent call last):
  File "/Users/admin/Documents/Coding/Python/Scrapping/UpWork/V2/Selenium_login-immo.py", line 15, in <module>
    driver.find_element_by_id("_submit").click()
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button type="submit" id="_submit" name="_submit" class="button-register no-margin-bottom large">...</button> is not clickable at point (429, 324). Other element would receive the click: <div id="tarteaucitronRoot" class="tarteaucitronBeforeVisible">...</div>
  (Session info: chrome=95.0.4638.69)

(vscrap) admin@Admins-MacBook-Pro V2 % /Users/admin/Documents/Coding/Python/Scrapping/vscrap/bin/python /Users/admin/Documents/Coding/Python/Scrapping/UpWork/V2/Selenium_login-immo.py
Traceback (most recent call last):
  File "/Users/admin/Documents/Coding/Python/Scrapping/UpWork/V2/Selenium_login-immo.py", line 15, in <module>
    driver.find_element_by_id("_submit").click()
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/Users/admin/Documents/Coding/Python/Scrapping/vscrap/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button type="submit" id="_submit" name="_submit" class="button-register no-margin-bottom large">...</button> is not clickable at point (429, 324). Other element would receive the click: <div id="tarteaucitronRoot" class="tarteaucitronBeforeVisible">...</div>
  (Session info: chrome=95.0.4638.69)

Why can't I click this Submit button?

Nimantha
  • 6,405
  • 6
  • 28
  • 69
lchan
  • 1

1 Answers1

1

The OK, accept all button is a dynamic element, so ideally to click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.tarteaucitronCTAButton.tarteaucitronAllow#tarteaucitronPersonalize2"))).click()
    
  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='tarteaucitronCTAButton tarteaucitronAllow' and @id='tarteaucitronPersonalize2']"))).click()
    
  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • thanks, i have added the missing import and the webDriverWait but still the same the cookies popup remains and the submit has not been clicked. – lchan Nov 14 '21 at 10:46
  • I added a short delay to it and then it worked. `w_item = WebDriverWait(driver,50).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.tarteaucitronCTAButton.tarteaucitronAllow#tarteaucitronPersonalize2"))) time.sleep(1) w_item.click()` – user3430832 Nov 29 '21 at 18:21