1

I am trying to change store location on a website with python and selenium. Selenium IDE addon from Firefox gives me a sequence. The sequence works within selenium IDE from Firefox but I cannot get it to work from within Python (Spyder). The element I want to click is inside a script and no tool can find an element inside a script. Beautifulsoup can not do it, neither can selenium. With the following code I try to get a product price in every store so I need to change the store (it is that yellow button in the top left corner and then the dropdown list) to every store from the dropdown list and scrape the page source for product price. But whenever I try "driver.find_element_by_" I get "Unable to locate element: "

The click sequence is recorded with Selenium IDE addon from Firefox. Or maybe there is a faster way than selenium to switch between shops and get the product price. I could not do it just with Beautifulsoup.

from selenium import webdriver
driver = webdriver.Firefox(executable_path='d:\Work\geckodriver.exe')
url = 'https://www.castorama.pl/deska-14x90x540-eslov-jodel-1-94-id-1105153.html'
driver.get(url)
driver.maximize_window()
driver.find_element_by_id("market-name").click() #Unable to locate element
driver.find_element_by_id("shop-selection-master-infostore").click()
driver.find_element_by_xpath("//div[@id='geolocation_popup_select_market_chosen']/a/span").click()
driver.find_element_by_xpath("//div[@id='geolocation_popup_select_market_chosen']/div/ul/li[2]").click()
DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Dictador
  • 181
  • 1
  • 5

1 Answers1

0

Dismiss the cookie bar:

from selenium import webdriver

driver = webdriver.Firefox(executable_path='d:\Work\geckodriver.exe')
url = 'https://www.castorama.pl/deska-14x90x540-eslov-jodel-1-94-id-1105153.html'
driver.get(url)
driver.maximize_window()
driver.find_element_by_css_selector('[onclick="bold.cookie.closeCookie();"]').click()
driver.find_element_by_id('shop-selection-master').click()

Purists won't like it but you can also use javascript

driver.execute_script("document.querySelector('#shop-selection-master').click();")

selecting options is covered extensively here: Selenium - Python - drop-down menu option value

You have the id of the parent select: geolocation-popup-select-market.

QHarr
  • 83,427
  • 12
  • 54
  • 101
  • After dismissing the cookie bar the line driver.find_element_by_id('shop-selection-master').click() still fails to find the element. However, "driver.execute_script" does the trick and brings up the popup window. I switch to it with "driver.switch_to.active_element; driver.find_element_by_xpath("//div[@id='geolocation_popup_select_market_chosen']/a/span").click();driver.find_element_by_xpath("//div[@id='geolocation_popup_select_market_chosen']/div/ul/li[5]").click() ". Then, the page configures itself to the selected shop and then I am unable to switch to the default content. – Dictador Jun 22 '19 at 09:39
  • You probably need to loop through setting the store each time – QHarr Jun 22 '19 at 09:43
  • Can the selection of the store (executing the script) and then scrapping the webpage content be done with PyQt? Selenium is painfully slow. – Dictador Jun 22 '19 at 10:15