How to extract data with selenium from a website?

Question

I'm trying to extract some data with selenium and python from a website. Here is some code I use until now:

url = 'https://www.kicker.de/1894559/spieldaten/bayern-muenchen-14/borussia-mgladbach-15'
chrome = webdriver.Chrome()
select = Select(chrome.find_element_by_id('selOptaPlayer1'))
select.select_by_value('57598')

It is possible to open the URL, select the player on the website. But I can't extract some data from the player. I get the error that selenium can't find the xpath. With google help, the issue can be with iframes. Now my question is, how can I find out in which iframe is the data located?

you can switch between iframe, take a look at [this](https://stackoverflow.com/questions/44834358/switch-to-an-iframe-through-selenium-and-python), then you can retry your find_by, till you find what you need — WoAiNii, Mar 26 '20 at 18:59
Please provide a [mcve], as well as the entire error message. — AMC, Mar 26 '20 at 19:18

score 0 · Accepted Answer · answered Mar 26 '20 at 19:15

Please find below solution to select 57598 value option using option[text()='Alaba, David']

Solution 1:

from selenium import webdriver
driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.get('https://www.kicker.de/1894559/spieldaten/bayern-muenchen-14/borussia-mgladbach-1')
driver.find_element_by_xpath("//select[@id='selOptaPlayer1']/option[text()='Alaba, David']").click()

Solution 2:

Also you can use below code for the same :

from selenium import webdriver
from selenium.webdriver.support.ui import Select


driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.get('https://www.kicker.de/1894559/spieldaten/bayern-muenchen-14/borussia-mgladbach-1')
select.select_by_value('57598')

How to extract data with selenium from a website?

1 Answers1