How do I limit Selenium Webdriver to view only rows made visible by a dynamic table

Question

I'm a bit out of my depth, but I'll do my best to explain. I'll put my question up top and then my explanation below:

"How do I tell Selenium to only read the results that are visible on the browser, and not what is held in the page source?"

I have a web page for which I've been creating automated tests using Selenium WebDriver. On the page there is a table of results, usually about 50 or so. When the user hits the page, all results are displayed. If they enter text into a search box, they can filter the results down. Nothing fancy. My goal is to ensure that the number of results returned by the table match the expected number.

The code that I've been using to count the number of results is this:

public int searchTableNumberOfMatchesOneColumn(String tableIdParam, int columnToSearchParam, String valueToFindParam){
      int numberOfMatchesFound = 0;

      WebElement table_element = driver.findElement(By.id(tableIdParam));
      List<WebElement> tr_collection=table_element.findElements(By.xpath("id('" + tableIdParam + "')/tbody/tr"));

      int row_num = 1;
      int col_num = 1;
      for(WebElement trElement : tr_collection)
      {
         List<WebElement> td_collection=trElement.findElements(By.xpath("td"));
         col_num=1;
         for(WebElement tdElement : td_collection)
         {
            if(col_num == columnToSearchParam && tdElement.getText().equals(valueToFindParam)){
               numberOfMatchesFound++;
            }
            col_num++;
         }
         row_num++;
      }

      return numberOfMatchesFound;
   }

The problem that I'm running into is this: This code always returns the FULL size of the table, 50 or so, and not the number of results visible on screen. I opened up the page source for the web page before and after searching to do a comparison, and they're identical. So while what is visible to the user changes, what is stored in the page source is not.

I've tried using a JavascriptExecutor to look for innerHTML, but it returns the same results. (As does doing driver.findElement(By.tagName("body")).getText();

Any help is appreciated.

Are you getting the expected count when you find the locator in developer tools? Can you post the url if it can be accessed by everyone? May be 'tdElement.isVisible()' before 'tdElement.getText' help. — Dhamo, Nov 07 '18 at 17:57
If you will post a link to the page with the table or at least post some sample rows of HTML, hidden and unhidden, I think I can significantly simplify this method. You probably really only need a single locator to find what you want and then filter that list down to only visible. It's probably a two-liner method but I need that info to write it. — JeffC, Nov 07 '18 at 19:28
OK ... I lied... it was a one-liner. :) I provided sample TABLE HTML and some example code that you can take and adapt. — JeffC, Nov 07 '18 at 19:46

score 0 · Answer 1 · answered Nov 07 '18 at 19:45

Well I'll just give you some sample HTML and the code that goes with it and you can adapt it yourself.

Sample table

<table id="myTable">
    <tr>
        <td>A1</td>
        <td>B1</td>
        <td>C1</td>
    </tr>
    <tr style="display:none">
        <td>A1</td>
        <td>B1</td>
        <td>C1</td>
    </tr>
</table>

Sample code

public int searchTableNumberOfMatchesOneColumn(String tableIdParam, int columnToSearchParam, String valueToFindParam)
{
    return driver.findElements(By.xpath("//table[@id='" + tableIdParam + "']//tr/td[" + columnToSearchParam + "][.='" + valueToFindParam + "']")).stream().filter(e -> e.isDisplayed()).collect(Collectors.toList()).size();
}

Basically this uses an XPath to search for a TABLE with the specified ID that contains a TD in column X (provided) that also contains text Y (also provided). This XPath will return ALL elements, not just the visible ones so we need to filter the list by what is visible. We can do that using Java stream() and filter(). See this link for more info, including the doc links inside.

I don't have an IDE handy so there may be some minor adjustments you will have to make but this should point you in the right direction. You should spend some time learning more about locators like CSS selectors and XPath. They are very powerful and will save you lots of extra code to find what you want by looping, etc. — JeffC, Nov 07 '18 at 19:48

score 0 · Answer 2 · answered Nov 08 '18 at 07:40

Simply use NOT operator in xpath.

suppose you have html table as follow

 <table id="myTable">
    <tr style="display:block">
        <td>A1</td>
        <td>B1</td>
        <td>C1</td>
    </tr>
    <tr style="display:none">
        <td>A1</td>
        <td>B1</td>
        <td>C1</td>
    </tr>
</table>

Following xpath will return all visible tr

//table[@id='myTable']//tr[not(@style='display:none')]

Sample Image

score 0 · Answer 3 · answered Nov 13 '18 at 11:54

The behavior you are observing that the ...code always returns the FULL size of the table, 50 or so, and not the number of results visible on screen... is pretty much the expected result.

Unless the Application Under Test (AUT) is implemented using Lazy Loading, it's a blessing in disguise if the entire table is present within the HTML DOM before the Client (i.e. the Web Browser) is returning back the control to the WebDriver instance on 'document.readyState' equal to "complete" being achieved.

You can find a relevant discussion in Do we have any generic funtion to check if page has completely loaded in Selenium

Element Visiblity

Element Visible implies that an element is present on the DOM of a page and visible. Visibility means that the element is not only displayed but also has a height and width that is greater than 0.

Note: Element Visible doesn't necessarily imply that the element is interactable/clickable.

Viewport

Viewport is the user's visible area of a web page. To interact or click an element, the element needs to be within the Viewport.

Solution

If you want to restrict the collection i.e. List with the elements only within the Viewport a possible solution will be to look at the following ExpectedConditions() classes:

numberOfElementsToBe()- An expectation for checking number of WebElements with given locator.
numberOfElementsToBeLessThan()- An expectation for checking number of WebElements with given locator being less than defined number
numberOfElementsToBeMoreThan()- An expectation for checking number of WebElements with given locator being more than defined number

How do I limit Selenium Webdriver to view only rows made visible by a dynamic table

3 Answers3

Element Visiblity

Viewport

Solution