Can't get element value using selenium when parsing SPA site

Question

Also, I can't get at all any of the attributes that I see on the object.

The HTML-code of the element I'm trying to get the value from:

<div data-bn-type="text" class="draggableHandle css-1nxx0n5">Target text</div>

My parser-code:

chromedriver = '/usr/bin/chromedriver'
opts = webdriver.ChromeOptions()
opts.add_argument('headless')
browser = webdriver.Chrome(options=opts, executable_path=chromedriver)
browser.get(url)

elem = browser.find_element(by=By.CLASS_NAME, value='draggableHandle')
print(elem) # value "<selenium.webdriver.remote.webelement.WebElement (session="9a2947e3e80269469c153808fbba18b6", element="77af1794-a2b0-4522-a261-f7211869e6c1")>"
print(elem.get_attribute('session')) # value "None"
print(elem.get_attribute('element')) # value "None"
print(elem.text) # null value

I didn't find an answer on Google. Please tell me what am I doing wrong.

I will be grateful for any answer!

you could try with the whole name of the class "draggableHandle css-1nxx0n5", or use xpath — Andres Ospina, Mar 12 '23 at 21:23
@AndresOspina Thanks for the answer! I'm trying to get by partial match because the part `css-1nxx0n5` is dynamic. It only has two values, but I don't want to make two requests, because this will slow down the script. By complete match it turns out to be found. Can you tell me how to set `xpath`? — Vsevolod, Mar 12 '23 at 21:33

score 0 · Answer 1 · answered Mar 12 '23 at 21:28

find_element() returns a webelement which doesn't have the following attribute/property:

session
element

Hence, None is printed.

Given the HTML:

<div data-bn-type="text" class="draggableHandle css-1nxx0n5">Target text</div>

To print the text Target text ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

Using CLASS_NAME and get_attribute("textContent"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CLASS_NAME, "draggableHandle"))).get_attribute("textContent"))

Using CSS_SELECTOR and text attribute:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.draggableHandle"))).text)

Using XPATH and get_attribute("innerHTML"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(@class, 'draggableHandle')]"))).get_attribute("innerHTML"))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

Can't get element value using selenium when parsing SPA site

1 Answers1