Im trying to scrape a javascript page using selenium and having some problems clicking through. The click does not go to another page but uses javascript to bring up the next ten reviews, which I want to scrape.
The first click seems to work but the second click never works, always saying there is no element exists.
the code im using is
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
browser = webdriver.Firefox()
browser.get("http://www.agoda.com/the-coast-resort-koh-phangan/hotel/koh-phangan-th.html")
delay = 3 # seconds
xpath = "//a[@id='next-page']"
try:
WebDriverWait(browser, delay).until(expected_conditions.element_to_be_clickable((By.XPATH, xpath)))
print "Page is ready!"
except TimeoutException:
print "Loading took too much time!"
browser.find_element_by_xpath("//a[@id='next-page']").click()
try:
WebDriverWait(browser, delay).until(expected_conditions.element_to_be_clickable((By.XPATH, xpath)))
print "Page is ready!"
except TimeoutException:
print "Loading took too much time!"
browser.find_element_by_xpath("//a[@id='next-page']").click()
try:
WebDriverWait(browser, delay).until(expected_conditions.element_to_be_clickable((By.XPATH, xpath)))
print "Page is ready!"
except TimeoutException:
print "Loading took too much time!"
which gives
Page is ready!
Page is ready!
WebDriverException: Message: Element is not clickable at point
Any ideas why this is not working, I have checked that the element to click is there.
What I dont understand is that it says the page is ready, therefore it has found the element I am trying to click but then when I go and click on this element it then says the element is not clickable?