0

I am new to Selenium and Python. I would like to navigate through a website, find an element and print it (or store it in a csv file).

Python version: 3.10; Selenium Webdriver: Firefox; IDE: PyCharm 2021.3.2 (CE); OS: Fedora 35 VM

So far I am able to navigate to the appropriate page where a table is generated. When I locate the element by CSS Selector and attempt to print it, the output does not print the text "$10.50" that I see on the screen. In fact, it does not print anything.

HTML code of the element I am trying to print:

<input id="b8-b36-Input_RemainAmtYr1" class="form-control OSFillParent" data-input="" disabled="" type="text" style="margin-top: 5px;" value="$10.50">
event

My relevant code:

RemainDue = driver.find_element(By.CSS_SELECTOR, '#b8-b36-Input_RemainAmtYr1')
print ('Remaining Due:', RemainDue.text)

I expect the output to be "$10.50", which is what I see on the screen and in the HTML. Instead I get the following:

Remaining Due:

There is nothing printed after "Remaining Due:" I thought the problem was that the "$10.50" was possibly being generated by some sort of javascript but I can see the text "$10.50" that I want to print in the HTML code above. What am I doing wrong?

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Auditor
  • 45
  • 1
  • 10
  • Looks like HTML id will change on every page load, can you share the URL or more HTML code for better answer. – Gaurav Lad Mar 11 '22 at 16:14

1 Answers1

2

To print the value of the value attribute i.e. $10.50 you can use either of the following locator strategies:

  • Using css_selector:

    print(driver.find_element_by_css_selector("input.form-control.OSFillParent[id$='Input_RemainAmtYr1']").get_attribute("value"))

  • Using xpath:

    print(driver.find_element_by_xpath("//input[@class='form-control OSFillParent' and contains(@id, 'Input_RemainAmtYr1')]").get_attribute("value"))

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input.form-control.OSFillParent[id$='Input_RemainAmtYr1']"))).get_attribute("value"))

  • Using XPATH:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//input[@class='form-control OSFillParent' and contains(@id, 'Input_RemainAmtYr1')]"))).get_attribute("value"))

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in Python Selenium - get href value

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • The "Using CSS_SELECTOR:" method worked! Thank you so much. Pycharm prompted me to make the following sight change: print(driver.find_element(By.CSS_SELECTOR, "input.form-control.OSFillParent[id$='Input_RemainAmtYr1']").get_attribute("value")) Not sure if it has to do with the version of Selenium I am using. I have been trying to figure this out how to print this out for over a week before I posted the question. Is there anywhere I can read up on how to print the value of value attributes? – Auditor Mar 11 '22 at 15:42
  • 1
    @Auditor Checkout the updated answer and let me know if you have any further queries. – undetected Selenium Mar 11 '22 at 15:49
  • The WebDriverWait code you provided does work if I replace 'visibility_of_element_located' with 'presence_of_element_located'. It does locate the element & I am able to extract the text attribute of that element. But I am concerned because apparently best use dictates that I should use 'visibility_of_element_located' if I want to extract any attribute of any element & I do need to extract the text attribute. Why would WebDriverWait work for me with 'presence_of_element_located' and not work with 'visibility_of_element_located'? The code is otherwise identical. – Auditor Mar 15 '22 at 21:50
  • I think I understand now. The element I am trying to extract the text attribute from is not visible on the page unless another tab is clicked. I didn't bother adding the Selenium code to click that tab first because the locate code was able to extract the text attribute prior to adding WebDriverWait to it. When I did add WebDriverWait, I was using 'visibility_of_element_located' even though that element was not visible. When I changed it to 'presence_of_element_located' I was able to extract the text attribute from it. – Auditor Mar 16 '22 at 00:46
  • Do you think it is okay to leave the code as 'presence_of_element_located' even though I am extracting text from it or should I change the code to click the tab first and then use 'visibility_of_element_located'? – Auditor Mar 16 '22 at 00:46
  • 1
    Ideally using [Selenium](https://stackoverflow.com/a/54482491/7429447) it is desireable to probe the [_visibility_of_element_located()_](https://stackoverflow.com/a/50474905/7429447) of an element. However, in real time scenarios at times we do have to compromise on the best practices and such cases it's okay to use [_presence_of_element_located()_](https://stackoverflow.com/a/57313803/7429447). – undetected Selenium Mar 16 '22 at 11:09