Scrapy + Selenium + Datepicker

Question

So i need to scrap a page like this for example and i am using Scrapy + Seleninum to interact with the datepicker calendar but i am running into a ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with.

So far i have:

def parse(self, response):

    self.driver.get("https://www.airbnb.pt/rooms/9315238")
    try:
        element = WebDriverWait(self.driver, 10).until(
            EC.presence_of_element_located((By.XPATH, "//input[@name='checkin']"))
        )
    finally:
        x = self.driver.find_element_by_xpath("//input[@name='checkin']").click()
        import ipdb;ipdb.set_trace()
        self.driver.quit()

I saw some references on how to achieve this https://stackoverflow.com/a/25748322/977622 and https://stackoverflow.com/a/19009256/977622 .

I appreciate if someone could help me out with my issue or even provide a better example on how i can interact the this datepicker calendar.

Has the answer helped? Thanks. – alecxe Apr 06 '16 at 16:04 — alecxe, Apr 06 '16 at 16:04
yes it did, sorry i took too long to mark it as correct – psychok7 Apr 06 '16 at 16:08 — psychok7, Apr 06 '16 at 16:08

score 1 · Accepted Answer · answered Mar 04 '16 at 15:39

There are two elements with name="checkin" - the first one that you actually find is invisible. You need to make your locator more specific to match the desired input. I would also use the visibility_of_element_located condition instead:

element = WebDriverWait(self.driver, 10).until(
    EC.visibility_of_element_located((By.CSS_SELECTOR, ".book-it-panel input[name=checkin]"))
)

Scrapy + Selenium + Datepicker

1 Answers1

Linked