Element is not currently visible and so may not be interacted with, Selenium Dropdown Box Python

Question

I need to get through all the pages on my webpage. There is a dropdown box in the left upper corner on all of these pages, with all the available cities. I want to visit every page, by choosing every position in this dropdown box. The dropdown box has a scrollbar, and when I want to choose the option which is below it, it gives me exception message:

Message: Element is not currently visible and so may not be interacted with
Stacktrace:
    at fxdriver.preconditions.visible (file:///tmp/tmpHWLMyH/extensions/fxdriver@googlecode.com/components/command-processor.js:9981)
    at DelayedCommand.prototype.checkPreconditions_ (file:///tmp/tmpHWLMyH/extensions/fxdriver@googlecode.com/components/command-processor.js:12517)
    at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmpHWLMyH/extensions/fxdriver@googlecode.com/components/command-processor.js:12534)
    at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmpHWLMyH/extensions/fxdriver@googlecode.com/components/command-processor.js:12539)
    at DelayedCommand.prototype.execute/< (file:///tmp/tmpHWLMyH/extensions/fxdriver@googlecode.com/components/command-processor.js:12481)

Heres the code :

#!/bin/env/python
# -*- coding: utf-8 -*-

import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

def get_browser():
    return webdriver.Firefox()


main_page_url = "http://example.com/"
basic_url = 'http://example.com/ogloszenia-kobiet.html'

def get_city_list(url) :

    AGE_ACCEPT_BUTTON_XPATH = ".//*[@id='columns']/div/div[2]/section/div/div/div/div/div/div[2]/div[2]/a[2]"
    COMBOBOX_XPATH = ".//*[@id='select_city']/li/form/div/button"
    COMBOBOX_OPTION_XPATH = ".//*[@id='select_city']/li/form/div/div/ul/li[%s]/a/span[1]"
    CHOOSE_BUTTON_XPATH = ".//*[@id='select_city']/li/form/button"

    pages = []

    try:
        browser = get_browser()
        wait = WebDriverWait(browser, 100)
        browser.get(main_page_url)
        time.sleep(2)

        button_age_accept = browser.find_element_by_xpath(AGE_ACCEPT_BUTTON_XPATH)
        button_age_accept.click()
        time.sleep(10)

        browser.get(url)

        i = 2

        while(True) :
            try :

                button_combobox = browser.find_element_by_xpath(COMBOBOX_XPATH)
                button_combobox.click()
                time.sleep(5)

                element_xpath = COMBOBOX_OPTION_XPATH % i
                option_in_combobox = browser.find_element_by_xpath(element_xpath)
                # wait.until(EC.invisibility_of_element_located((By.XPATH, element_xpath)))
                # option_in_combobox = WebDriverWait(browser, 10).until(lambda browser : browser.find_element_by_xpath(element_xpath))
                option_in_combobox.click()
                time.sleep(5)

                button_choose = browser.find_element_by_xpath(CHOOSE_BUTTON_XPATH)
                button_choose.click()
                time.sleep(5)

                pages.append(browser.current_url)

                i += 1
            except Exception, e:
                print e
                break

        browser.close()
        return pages

    except Exception, e:
        info = 'Generic exception\n'
        print e
        return []

get_city_list(basic_url)

I also tried with scroll bar, tried to move it down, but still, no effect. I can scroll only the pages that are at the top of this drop down box:

#!/bin/env/python
# -*- coding: utf-8 -*-

import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

def get_browser():
    return webdriver.Chrome()


main_page_url = "http://example.com/"
basic_url = 'http://example.com/ogloszenia-kobiet.html'

def get_city_list(url) :

    AGE_ACCEPT_BUTTON_XPATH = ".//*[@id='columns']/div/div[2]/section/div/div/div/div/div/div[2]/div[2]/a[2]"
    COMBOBOX_XPATH = ".//*[@id='select_city']/li/form/div/button"
    COMBOBOX_OPTION_XPATH = ".//*[@id='select_city']/li/form/div/div/ul/li[%s]/a/span[1]"
    CHOOSE_BUTTON_XPATH = ".//*[@id='select_city']/li/form/button"

    pages = []

    try:
        browser = get_browser()
        wait = WebDriverWait(browser, 100)
        browser.get(main_page_url)
        time.sleep(2)

        button_age_accept = browser.find_element_by_xpath(AGE_ACCEPT_BUTTON_XPATH)
        button_age_accept.click()
        time.sleep(10)

        browser.get(url)

        i = 2

        while(True) :
            try :

                button_combobox = browser.find_element_by_xpath(COMBOBOX_XPATH)
                button_combobox.click()
                time.sleep(5)

                element_xpath = COMBOBOX_OPTION_XPATH % i
                option_in_combobox = browser.find_element_by_xpath(element_xpath)

                actionChains = ActionChains(browser)
                scrollbar = browser.find_element_by_xpath("/html/body/section/section[2]/div/div[2]/section/div/div/div/div[1]/ul/li/form/div/div/ul")
                actionChains.click_and_hold(scrollbar).perform()
                actionChains.move_by_offset(0,10+i).perform()
                actionChains.release()

                browser.execute_script("arguments[0].scrollIntoView();", option_in_combobox)
                option_in_combobox.click()

                browser.execute_script("window.scrollTo(0, 0);")

                button_choose = browser.find_element_by_xpath(CHOOSE_BUTTON_XPATH)
                button_choose.click()
                time.sleep(5)

                pages.append(browser.current_url)

                i += 1
            except Exception, e:
                print e
                break

        browser.close()
        return pages

    except Exception, e:
        info = 'Generic exception\n'
        print e
        return []

pages = get_city_list(basic_url)
for p in pages :
    with open('links.txt', 'a') as the_file:
        the_file.write(p)
        the_file.write('\n')

------------------------------------------------------------------------------------------------------------------------------------ UPDATE: ------------------------------------------------------------------------------------------------------------------------------------

For now on, I'm using Kubuntu 14.04. I have Python 2.7.11 and Selenium 2.49.2. My current code:

#!/bin/env/python
# -*- coding: utf-8 -*-

import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

# def get_browser():
#     options = webdriver.ChromeOptions()
#     options.add_argument("--start-maximized")
#     return webdriver.Chrome(chrome_options=options)


def get_browser():
    return webdriver.Firefox()

main_page_url = "http://example.com/"
basic_url = 'http://example.com/ogloszenia-kobiet.html'

def get_city_list(url) :

    AGE_ACCEPT_BUTTON_XPATH = ".//*[@id='columns']/div/div[2]/section/div/div/div/div/div/div[2]/div[2]/a[2]"
    COMBOBOX_XPATH = ".//*[@id='select_city']/li/form/div/button"
    COMBOBOX_OPTION_XPATH = ".//*[@id='select_city']/li/form/div/div/ul/li[%s]/a/span[1]"
    CHOOSE_BUTTON_XPATH = ".//*[@id='select_city']/li/form/button"

    pages = []

    try:
        browser = get_browser()
        wait = WebDriverWait(browser, 100)
        browser.get(main_page_url)
        time.sleep(2)

        button_age_accept = browser.find_element_by_xpath(AGE_ACCEPT_BUTTON_XPATH)
        button_age_accept.click()
        time.sleep(10)

        browser.get(url)

        i = 2

        while(True) :
            try :

                button_combobox = browser.find_element_by_xpath(COMBOBOX_XPATH)
                button_combobox.click()
                time.sleep(5)

                element_xpath = COMBOBOX_OPTION_XPATH % i
                option_in_combobox = browser.find_element_by_xpath(element_xpath)
                option_in_combobox.click()

                button_choose = browser.find_element_by_xpath(CHOOSE_BUTTON_XPATH)
                button_choose.click()
                time.sleep(5)

                pages.append(browser.current_url)

                i += 1
            except Exception, e:
                print e
                break

        browser.close()
        return pages

    except Exception, e:
        info = 'Generic exception\n'
        print e
        return []

pages = get_city_list(basic_url)
for p in pages :
    with open('links.txt', 'a') as the_file:
        the_file.write(p)
        the_file.write('\n')

For Firefox, the code exits at element 'Gdańsk' with a message: string indices must be integers, so it means it does not find every element in my combobox.

For Chrome and Windows XP, it exits at 'Bielsko-Biała', so again, it means it does not find every element in my combobox. ...

How can I solve this issue?

Remembering that this website has users as young as 13 as well as professionals accessing the site from work, please don't link to websites containing nudity in your question. Include enough details in the question itself so that the question can be answered. — josliber, Feb 07 '16 at 02:19

score 4 · Answer 1 · edited May 23 '17 at 12:17

4

Either move to element and then click:

from selenium.webdriver.common.action_chains import ActionChains

actions = ActionChains(browser)
actions.move_to_element(option_in_combobox).click().perform()

or, scroll into it's view:

browser.execute_script("arguments[0].scrollIntoView();", option_in_combobox)

or, click the element via javascript:

browser.execute_script("arguments[0].click();", option_in_combobox)

For Firefox, the code exits at element 'Gdańsk' with a message: string indices must be integers, so it means it does not find every element in my combobox.

There is an existing problem in selenium 2.49 that may cause this TypeError. You would need to downgrade to 2.48:

pip install selenium==2.48

Python & Selenium Click and "TypeError: string indices must be integers"

edited May 23 '17 at 12:17

Community

1
1

answered Jan 24 '16 at 20:26

alecxe

462,703
120
1,088
1,195

First of all: thank you for your answer. Second: I modified my code, as you suggested, and 1) your second approach (this code: https://gist.github.com/anonymous/00789e9cc3c13fc6477c) gives me `Element is not clickable at point ... ` and does not find all the links I need, just some of them 2) your first approach (my code: https://gist.github.com/anonymous/880841d0166943133a42) gives `element not visible` and finds no links at all 3) your third approach (my code: https://gist.github.com/anonymous/6261a924b4f15760742f) also gives `element not visible` and any links are saved ... any ideas? – Katie Jan 24 '16 at 21:47
@Katie please first try the Cyril's answer and see if it helps. Let's see if the problem would still exist. Thanks. – alecxe Jan 25 '16 at 02:39
@Katie please see the update. It should explain your string indices must be integers error. – alecxe Jan 30 '16 at 16:18

score 3 · Answer 2 · answered Jan 24 '16 at 23:50

3

Like mentioned you should first move to the option element before clicking on it: actions.move_to_element(option_in_combobox).click().perform(). Now is the 'button_choose' element not visible because the browser scrolled down. To fix this you need scroll to the top and then click on the button:

browser.execute_script("window.scrollTo(0, 0);") # scroll to top

button_choose = browser.find_element_by_xpath(CHOOSE_BUTTON_XPATH)
button_choose.click()

answered Jan 24 '16 at 23:50

Cyril

2,376
16
21

This also does not work. My code: http://codepad.org/Utw8X1Dw gets only some of the available links, not all of them. It seems it still has a problem with scrolling to the cities that are at the bottom of this drop down box. – Katie Jan 25 '16 at 12:24
Your code works for me. I set i to 40 and it worked for all the cities at the bottom (including 'zielona-gora'). Note: I had to maximize Chrome to get it to work, Firefox worked out of the box. – Cyril Jan 25 '16 at 18:06
This is actually weird. Tried my code both on Kubuntu and Windows, and have same errors, example errors for the code I posted (tested on Windows): https://gist.github.com/anonymous/8a6d50228ac12cc4cc72, https://gist.github.com/anonymous/eac439ef03ce9509e776, modified code: https://gist.github.com/anonymous/a0fc50801436948bbf5c – Katie Jan 25 '16 at 20:24
This is weird indeed. I just ran the gist and it worked as expected (Selenium 2.49.2 / Python 2.7 / Ubuntu 14.04) – Cyril Jan 25 '16 at 20:48
I tested it on my Kubuntu 14.04. I have Python 2.7.11 and Selenium 2.49.2 (I updated from 2.48 in a meanwhile). But now, the code gives me such a message `string indices must be integers` – Katie Jan 25 '16 at 21:30
Are you sure that there are actually links missing in the links.txt file? It fails to find the element at the end because there are no more options left (i > combobox size). But if it's not that im clueless. – Cyril Jan 26 '16 at 20:31
The code I posted finds only links for 'Gdynia' (and ends), which for sure is not the last element in combobox (the last element is 'Zielona Góra'), so it seems, it finds elements up to the VISIBLE element only, no matter if I use scrolling, as suggested by @alecxe. – Katie Jan 30 '16 at 09:50

score 3 · Answer 3 · edited Feb 07 '16 at 02:19

I tried in java and its working very fine. I request to please look on for loop mainly as here i am able to select options one by one. I just used Thread.sleep, as i know we can use waits also. To make similar to question i just appended to StringBuffer here, i know we can added to any collectors in java. I am not followed exactly as accepting age etc. as i said i concentrated on that drop down selection..

 public class Dog {

  private WebDriver driver;
  private String baseUrl;
  private StringBuffer verificationErrors = new StringBuffer();

  @BeforeClass()
  public void setUp() throws Exception {
   // driver = new FirefoxDriver();
      System.setProperty("webdriver.chrome.driver", "E:\\selenium_setups\\chromedriver_win32\\chromedriver.exe");
        driver=new ChromeDriver();
    driver.manage().window().maximize();
    baseUrl = "http://example.com/";
    driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
  }

  @Test
  public void testStackoverflowIssue() throws Exception {

   driver.get(baseUrl);
   driver.findElement(By.cssSelector(".btn.btn-success")).click();
   driver.get("http://example.com/ogloszenia-kobiet.html");
    List<WebElement> options=driver.findElements(By.xpath(".//*[@id='select_city']/li/form/div/div/ul/li"));

    for(int i=1; i<=options.size(); i++){

    driver.findElement(By.xpath("(//button[@type='button'])[4]")).click();
    driver.findElement(By.xpath("//ul[@id='select_city']/li/form/div/div/ul/li["+i+"]")).click();
    driver.findElement(By.cssSelector("button.btn.btn-success")).click();
    Thread.sleep(5000);
    verificationErrors.append(driver.getCurrentUrl());
    System.out.println(driver.getCurrentUrl() +" >>> iteration " +i);
    driver.navigate().to("http://example.com/ogloszenia-kobiet.html");
    Thread.sleep(5000);
  }
  }

}

I hope this will helps you in python to select drop down values..

output

Thank You, Murali

Thank you. I downgraded to `Selenium 2.48`, as suggested by @alecxe, and modified your code (translated it to Python actually), but the problem remains the same. My code gets only the visible part of the drop down box, however, when I maximize browser's window in Firefox, it gives me more links, than previous approaches (up to 'Krakow`). But then, after the 'Krakow' is reached, program ends, and its not possible to get the elements from the bottom of the drop down box. My modified code is here: https://gist.github.com/anonymous/9cb82acf839d6b92731d — Katie, Jan 30 '16 at 21:50
Since I need to build a Windows executable with this code, I also tried it with Windows XP and Chrome. There, the problem is exact as before. Your code can reach only 'Bielsko-Biala' and ends, saying that `Element is not clickable at point (1018, 508)` ... Huh — Katie, Jan 30 '16 at 21:56
@Katie, changed chromedriver. its worked very well see output image. I used this xpath //ul[@id='select_city']/li/form/div/div/ul/li["+i+"] to select values in drop down. i am not used up to span. i hope you are noted this. thanks — murali selenium, Jan 31 '16 at 02:35
I have the latest chromedriver ... anyway, this code does not work, at least on my computer, and my friend's computer ... — Katie, Feb 04 '16 at 21:12

score 3 · Answer 4 · answered Feb 05 '16 at 18:50

3

try this before selecting the "Not Currently invisible Element", and after clicking on the Dropdown menu:

browser.execute_script("document.getElementsByClassName('dropdown-menu inner selectpicker')[0].setAttribute('style', '');")

answered Feb 05 '16 at 18:50

Soufiaane

1,737
6
24
45

Element is not currently visible and so may not be interacted with, Selenium Dropdown Box Python

4 Answers4