Scraping first_selected_option from a select tag within gridcell name inside td tag

Question

I'm trying to scrape a table from a dynamic website. I'm using selenium to get into the right data and then using beautifulsoup to scrape the table but not able to get data from select tags. Like one select tag have two options Active and Cancelled but when I'm scraping this tag, the results shows both the text, instead I want only selected text like Active or Cancelled. Here is the html code.

<td _ngcontent-c3="" class="sticky_table_fourth_left_entry"><!----><!----><!----><!----><select _ngcontent-c3="" class="form-control status-select ng-untouched ng-pristine" disabled="" style="background-color: rgb(255, 96, 96);"><!----><option _ngcontent-c3="" value="Active">Active</option><option _ngcontent-c3="" value="Cancelled">Cancelled</option></select><!----><!----><!----><!----><!----><!----><!----><!----></td>

There isn't any web element for selected text but when I go to Accessibility tree then there is that selected text in gridcell.

Here is the image:

.

After doing a lot of research I still cannot find how to scrape that gridcell text, I don't know if there's any way to do that but if there is then please help here.

Answer 1

To scrape the text of the selected option you need to induce WebDriverWait for the visibility_of_element_located() and you have to use the first_selected_option attribute as follows:

select = Select(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//select[@class='form-control status-select ng-untouched ng-pristine']"))))
element = select.first_selected_option
print(element.text)

In a single line:

print(Select(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//select[@class='form-control status-select ng-untouched ng-pristine']")))).first_selected_option.text)

Note : You have to add the following imports :

from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC