0

I'm trying to scrape a robo advisor website using Selenium in Python. The problem I'm having is that I cannot select and click the elements to answer the required questions despite having tried selection with both CSS selectors and XPATH. I think there is something weird going on here that I'm missing so any help would be greatly appreciated.

The code snippet below shows how to get the survey and attempts to click on the first answer in the time horizon slider. Unfortunately this throws the error: selenium.common.exceptions.NoSuchElementException: Message: no such element.

Sorry the site is in German but if you have any questions just let me know!

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from time import sleep

# Load Selenium Webdriver
driver = webdriver.Chrome(r'C:\Users\jcgou\PycharmProjects\Master Thesis Robo Advisors\Chrome Driver\chromedriver.exe')

# Navigate to Fintego
fintegoUrl = 'https://www.fintego.de/'
driver.set_page_load_timeout(10)
driver.get(fintegoUrl)
driver.maximize_window()

# Navigate to portfolio recommendation survey
driver.find_element_by_link_text('Depot eröffnen').click()
sleep(3)

# Time Horizon
driver.find_element_by_xpath("//ul[@id='eox_ContentPane_3_Layout_AnlagehorizontBlock_lstAnlagehorizont']//li[2]//i[1]").click()

Here is the HTML if that helps at all. I'm trying to select and click on "1 - 3 Jahre".

<ul class="form name m-deo-form marginBottomDouble">
    <li id="eox_ContentPane_3_Layout_AnlagehorizontBlock" class="marginBottom">
        <label for="eox_ContentPane_3_Layout_AnlagehorizontBlock_lstAnlagehorizont" class="left selectList-label">
            <span id="eox_ContentPane_3_Layout_AnlagehorizontBlock_RequiredfieldvalidatorErfahrung" title="Bitte treffen Sie eine Auswahl" class="validator icon-exclamation" style="color:Red;visibility:hidden;"></span>
        </label>
        <div class="right">
            <ul id="eox_ContentPane_3_Layout_AnlagehorizontBlock_lstAnlagehorizont" class="selectList">
                <li class="item stopLightboxSehrKurzfristig" data-value="1" style="width:25%;"><span class="text">Kürzer als 1 Jahr</span><i></i></li>
                <li class="item" data-value="2" style="width:25%;"><span class="text">1 - 3 Jahre</span><i></i></li>
                <li class="item" data-value="3" style="width:25%;"><span class="text">3 - 7 Jahre</span><i></i></li>
                <li class="item" data-value="4" style="width:25%;"><span class="text">Länger als 7 Jahre</span><i></i></li>
            </ul><input type="hidden" name="eox_ContentPane_3$Layout$AnlagehorizontBlock$lstAnlagehorizont_ValueContainer" id="eox_ContentPane_3_Layout_AnlagehorizontBlock_lstAnlagehorizont_ValueContainer">
        </div>
    </li>
</ul>
Oleksandr Makarenko
  • 779
  • 1
  • 6
  • 18
Gore
  • 3
  • 3
  • Possible duplicate of [How do I switch to the active tab in Selenium?](https://stackoverflow.com/questions/28715942/how-do-i-switch-to-the-active-tab-in-selenium) – Guy Oct 15 '18 at 12:25
  • 1
    What does _"this does not work."_ mean? Does it click the wrong thing? Does it click the right thing but nothing happens? Does it throw an error, and if so, what is the error? – Bryan Oakley Oct 15 '18 at 12:47
  • Sorry about this! It throws the error: selenium.common.exceptions.NoSuchElementException: Message: no such element – Gore Oct 15 '18 at 12:50

2 Answers2

1

Try this, it worked for me in Chrome. You may want to replace the horrible time.sleep implementation with explicit wait. And I'd to switch to a new window to lookup the elements to click (may be this is why your code is not working)

from selenium.webdriver import Chrome
import time

driver = Chrome()
driver.get('https://www.fintego.de/')

# Navigate to portfolio recommendation survey
driver.find_element_by_link_text('Depot eröffnen').click()
time.sleep(10)

# Switch to new window
driver.switch_to.window(driver.window_handles[1])
anlage_horizon =  driver.find_element_by_id(
                    'eox_ContentPane_3_Layout_AnlagehorizontBlock_lstAnlagehorizont'
                  )

for item in anlage_horizon.find_elements_by_class_name('item'):
    if item.text == '1 - 3 Jahre':
        item.click()
Satish
  • 1,976
  • 1
  • 15
  • 19
  • Wow thank you so much this worked for me!! I will definitely be using this logic for the rest of the scraper. I wonder why you have to switch to a new window to have it work... – Gore Oct 15 '18 at 14:29
  • 1
    Glad it worked. The page opened in a new window and as far as I know, you need to explicit switch to the new window to work on the newly opened page – Satish Oct 15 '18 at 15:09
0

simply go grab element and click on it,

driver.find_element_by_xpath("*//span[text()='1 - 3 Jahre']").click();

this would work only if is clickable in the your HTML DOM (I hope it is).

Ashish Kamble
  • 2,555
  • 3
  • 21
  • 29
  • Unfortunately I did try doing this. It doesn't work and throws error mentioned in the comments above. – Gore Oct 15 '18 at 12:52