3

I am very new to Python and learning how to scrap data with Selenium.

I encounter a problem when trying to pick a date from a datepicker form on monmondo.com (for the sake of example)

This is the farthest I managed to get: (edit: I managed to go a little further than before but I am still stuck)

from selenium import webdriver
browser = webdriver.Firefox()

browser.get("https://www.momondo.com")
browser.implicitly_wait(5)
date = browser.find_element_by_name("ctl00$Content$ctl04$SearchFormv8$SearchFormFlight$InputDepart").click()
browser.implicitly_wait(5)
test= browser.find_elements_by_xpath("//*['ui-datepicker-div']//td[@data-year='2017'][@data-month='2']/a[@class='ui-state-default'][@href='#'][text()='20']")
test[0].click()

Which results in

selenium.common.exceptions.ElementNotVisibleException: Message:

I've tester the xpath with firepath and it seems to work correctly as it is found in the page's source code.

The webpage structure of the calendar's day in the source code is:

<td class=" " data-handler="selectDay" data-event="click" data-month="2" data-year="2017"><a class="ui-state-default" href="#">20</a></td>

    <a class="ui-state-default" href="#">20</a>

My vague guess is that the data-even click triggers the selection but it seems to be located a step above the class where I can find the number. This being said I am not sure it's the case.

I would really appreciate if you could help a newcomer like me!

Thanks!

François T
  • 43
  • 1
  • 4
  • ElementNotVisibleException, well, the element is not visible. Can you see that element in the browser? – Usmiech Feb 12 '17 at 17:10
  • Being said like that it sounds pretty straightforward! The code click to open the calendar but then nothing happens unfortunately. May it be something to do with the popup? Thanks a lot for your input. – François T Feb 12 '17 at 17:12
  • So that picker is in a popup window? If not, and you cannot click that in selenium, then maybe use something from jquery like $("css_selector").click() Selenium can execute JS. This should not fire errors, but make sure to check the developer console in the browser. Btw. monmondo.com shows some japanese hosting website – Usmiech Feb 12 '17 at 17:19
  • You are on another level! Yes indeed, it's a popup, thanks for the info, from now I am not quite sure how to use jquery in python code but I will look into it. Is there something special with japanese hosting website? I can try on another one, it's just for practice purpose. – François T Feb 12 '17 at 17:25
  • Check that http://stackoverflow.com/questions/17676036/python-webdriver-to-handle-pop-up-browser-windows-which-is-not-an-alert Look for something like "selenium popup window python". – Usmiech Feb 12 '17 at 17:33
  • And well, you can execute any js by browser.execute_script("somejs") – Usmiech Feb 12 '17 at 17:33
  • Thanks a lot!! I will duely look into all this, now I just need to let my brain cool down a little ;-) – François T Feb 12 '17 at 17:41

1 Answers1

3

Try to add some time to wait until element become visible:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Firefox()  
browser.get("https://www.momondo.com")
browser.implicitly_wait(5)
# Click to open drop-down
date = browser.find_element_by_xpath("//div[@class='input _date-depart']/div[@class='ui-calendar']/input").click()
# Choose depart date
wait(browser, 10).until(EC.visibility_of_element_located((By.XPATH, "//td[@data-handler='selectDay']/a[text()='20']"))).click()
# Choose return date
wait(browser, 10).until(EC.visibility_of_element_located((By.XPATH, "//td[@data-handler='selectDay']/a[text()='30']"))).click()
Andersson
  • 51,635
  • 17
  • 77
  • 129
  • Thanks a lot for this piece of code! At first when I ran it, it wait until timeout to open the popup, so to double check I ran it on another computer and there, miracle, it worked! I think it's related to a message I accidently discarded on a first run of Selenium, the issue is that I don't know how to correct it but it's another story. – François T Feb 13 '17 at 21:13
  • 1
    Welcome. If my answer helped you to resolve this current issue, please mark it as "Accepted". Thanks – Andersson Feb 13 '17 at 21:15
  • That's done, thank again! If I find the solution with Firefox I'll post it, so far I deleted Geckodriver and the pref files and it's still not working. I'll try to swipe it clean. – François T Feb 13 '17 at 21:36