0

Hi I am looking for an explanation on how selenium searches for an element on a website. For us, we use inspect element to find the id, name, xpath, etc. of an element, and then put it in selenium for some action to be done. How does selenium find the element we tell it to find? Does it ctrl shift J like us and inspect element?

Note: I am not looking for how to code selenium to find element.

hbae
  • 91
  • 7

3 Answers3

2

Html pages are just documents with elements structured like in a tree.

In general

Selenium uses element locators to find things. Locators work lazily. When you look up an element Selenium first checks if its cached. If not, it uses SearchContext which finds all elements within the current context (eg. DOM element) using a given mechanism, for example by XPathEvaluator.

SearchContext runs findElement() if you are looking for one element or findElements() if you are looking for more than one. In simple terms, findElement() tries to run JavaScript script to find the element asynchronously. If it can’t, it tries to find it directly by using an interestingly called method – xpathWizardry, i.e. by using XPathEvaluator evaluation.

XPath

When you use XPath (XML Path Language) in Selenium, this is just a way to navigate through hierarchical structure of an XML-like document, such as html. XPath uses a non-XML syntax to provide a flexible way of pointing to different parts of an XML document. Internally selenium uses W3 XPathEvaluator, which evaluates XPath expressions.

You can study XPathEvaluator source code here.

Pawel Kam
  • 1,684
  • 3
  • 14
  • 30
0

Search Context

The SearchContext is a topmost interface present in the Selenium WebDriver hierarchy. It has two methods that will be the abstract as SearchContext is an interface.

  • findElement(): Find the first WebElement using the given method.

    WebElement findElement​(By by)

Parameters:
by - The locating mechanism

Returns:
The first matching element on the current context

Throws:
NoSuchElementException - If no matching elements are found

  • findElements(): Find all elements within the current context using the given mechanism.

    java.util.List<WebElement> findElements​(By by)

Parameters:
by - The locating mechanism to use

Returns:
A list of all WebElements, or an empty list if nothing matches
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
0

The browser DOM exposes API like querySelector,querySelectorAll, getElementById, getElementsByClassName, getElementsByName, etc through javascript that can be used to locate elements. For example :

  1. Navigate to www.bing.com
  2. Press F12 to open developer console.
  3. Enter document.querySelector("#sb_form_q") to locate search box input by css selector. I am using #Id here as css selector.
  4. Enter document.getElementById("sb_form_q") to locate search box input by it's Id
  5. Enter document.getElementsByClassName("sb_form_q")[0] to locate search box input by it's class name
  6. Enter document.getElementsByName("q")[0] to locate search box input by it's name

All of above should return "<input id="sb_form_q" class="sb_form_q" name="q" type="search" maxlength="1000" autocomplete="off" aria-label="Enter your search term" autofocus="" aria-controls="sw_as" aria-autocomplete="both" aria-owns="sw_as" aria-activedescendant="sa_5004">" same result.

Selenium uses these DOM API to retrieve the elements. However, selenium might use these DOM API via some other mechanism (e.g C ++) and not by executing javascript for faster execution. XPath lookup is something not supported directly by browser DOM API. Selenium probably provides it's own implementation for XPath lookup or rely on some browser polyfill for this functionality.

Nish26
  • 969
  • 8
  • 16