1

I am using webdriver.Chrome (Selenium), to get links that are stored under an svg interactive map, at the following URL: https://www.mpcb.gov.in/water-quality

I have tried to use different functions of find_elements_by_ like xpath, partial_text, and more but without success. This is the beginning of my code:

DRIVER_PATH = 'C:/Users/Asaf/Desktop/Asaf/Python/chromedriver.exe'
options = Options()
options.headless = True
options.add_argument("--window-size=1920,1200")
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
 
# Site URL
url='https://www.mpcb.gov.in/water-quality'
driver.get(url)

To put it in context, I want to get these links (the clickable arena of the map) in order to scrape data from them, one at a time.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Asaf Pras
  • 13
  • 3

1 Answers1

0

To get the links from a svg interactive map, open the links in a new tab to scrape data from them, one at a time you can use the following Locator Strategies:

  • Code Block:

    driver.get('https://www.mpcb.gov.in/water-quality')
hrefs = [my_elem.get_attribute("xlink:href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div[property='schema:text'] svg g a")))]
parent_window  = driver.current_window_handle
for href in hrefs:
    driver.execute_script("window.open('" + href +"')")
    WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2))
    windows_after = driver.window_handles
    new_window = [x for x in windows_after if x != parent_window][0]
    driver.switch_to_window(new_window)
    print(driver.current_url)
    driver.close()
    driver.switch_to_window(parent_window)
driver.quit()

  • Console Output:

    https://www.mpcb.gov.in/water-quality/Thane/15
https://www.mpcb.gov.in/water-quality/Nashik/18
https://www.mpcb.gov.in/water-quality/Aurangabad/19
https://www.mpcb.gov.in/water-quality/Amravati/21
https://www.mpcb.gov.in/water-quality/Raigad/14
https://www.mpcb.gov.in/water-quality/Pune/17
https://www.mpcb.gov.in/water-quality/Nagpur/20
https://www.mpcb.gov.in/water-quality/Chandrapur/23
https://www.mpcb.gov.in/water-quality/Kolhapur/22
https://www.mpcb.gov.in/water-quality/Mumbai/12

References

You can find a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352