Use Selenium to extract all list items in an Unordered list

Question

I'm trying to create a Selenium Python script that finds a location on a map website (openstreetmap.org to be specific) according to input from the user (i.e. "Moscow"), then chooses a specific graphic layer for the map from the horizontal dropdown, also based on user input.

My script enters the website, opens the dropdown menu and finds the element containing the layer options, which is an unordered list.

However, I find that only 2 of the 5 layer options are being extracted. More so, the first extracted list item is the third option on the website and the second is the first option.

Here is my code. Entering openstreetmap.org:

base_url = 'https://www.openstreetmap.org/search?query=' + place_name
browser = webdriver.Firefox()
browser.get(base_url)
browser.implicitly_wait(3)

Opening the horizontal dropdown:

link_to_buttons = browser.find_elements_by_css_selector('a.control-button')
link_to_buttons[3].click()

Extracting the unordered list:

link_to_options = browser.find_elements_by_css_selector('ul.list-unstyled')

what's your issue and what's your desired output? – Alin Stelian Sep 21 '20 at 19:41 — Alin Stelian, Sep 21 '20 at 19:41

score 0 · Answer 1 · answered Sep 21 '20 at 19:54

0

You need to change locator.

ul.list-unstyled finds whole layer list (ul element), but you need options in this list (li element).

Let's try this locotor: .base-layers li

answered Sep 21 '20 at 19:54

Aliaksandr Plekhau

453
4
12

score 0 · Accepted Answer · answered Sep 21 '20 at 21:01

To extract the texts e.g. Standard, Cycle Map, etc from all of the <ul> using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get('https://www.openstreetmap.org/search?query=Moscow#map=10/55.7252/37.6290')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='control-button' and @data-original-title='Layers']/span"))).click()
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.section.base-layers>ul.list-unstyled li label")))])

Using XPATH:

driver.get('https://www.openstreetmap.org/search?query=Moscow#map=10/55.7252/37.6290')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.control-button[data-original-title='Layers']>span"))).click()
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='section base-layers']/ul[@class='list-unstyled']//li//label")))])

Console Output:

['Standard', 'Cycle Map', 'Transport Map', 'ÖPNVKarte', 'Humanitarian']

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Use Selenium to extract all list items in an Unordered list

2 Answers2