List not printing all .text values using Selenium and Python

Question

So I've been using selenium recently and decided to pull some values into a list using the following:

Values = browser.find_elements_by_xpath("//div[@class='main-col']//li[@class='test-item test-item--favourites']")

The values are all stored as expected. Though when I use for example:

for i in range (len(Values)):
    print(Values[i].text)
    i+=1

The code runs up to about the 50th index in the list and outputs the details, but after this its blank. I can run:

print(values[50])

It will output but .text gives me nothing.

Im using pYcharm for the development and looked into the variables at this stage. I can see that once I click on the variable it states 'collecting variable data' and after this I can then output some more of the variables in the list.

Is there a way I can force selenium to collect all the variable data so I can loop through each index in the list and store it in the .text format or is there a better alternative/method I can use?

Answer 1

If the .text is blank that means that there is no text between CSS Selector. To get an attribute out of a CSS Selector you have to use the get_attribute() function. Inside the () you place a string which is the name of an attribute which value you are looking for. for example

print(i.get_attribute('class'))

Answer 2

You are able to extract the text till the 50th index as find_elements_by_xpath() was able to identify 50 odd elements as per your Locator Strategy.

---

Solution

To collect all the desired elements using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR and get_attribute("innerHTML"):

  print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.main-col li.test-item.test-item--favourites")))])
  

• Using XPATH and text attribute:

  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='main-col']//li[@class='test-item test-item--favourites']")))])
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC