29

I am using Java Selenium project for web page automation. The web page contains lots of multi-level shadow-root DOM elements that I am not able to interact with using selenium findElement method.

I have tried the following solutions:

  • deep css (Don't work on latest chrome browser)
  • JS Executor. (This is really tedious and becomes complex to maintain)

Note:

If you know any other solution other than listed above that I can implement in Selenium Java framework , please pass on the solution. Thanks in advance !.

SushilG
  • 655
  • 1
  • 6
  • 19

6 Answers6

10

There is a very good plugin that can be used with selenium project shadow-automation-selenium. It helps in writing much better, readable and maintainable code. Using this you can access multi level of shadow DOM (up to 4 levels). This uses simple css selector to identify elements.

WebElement findElement(String cssSelector) : use this method if want single element from DOM

List<WebElement> findElements(String cssSelector) : use this if you want to find all elements from DOM

WebElement findElements(WebElement parent, String cssSelector) : use this if you want to find a single elements from parent object DOM

List<WebElement> findElements(WebElement parent, String cssSelector) : use this if you want to find all elements from parent object DOM

WebElement getShadowElement(WebElement parent,String selector) : use this if you want to find a single element from parent DOM

List<WebElement> getAllShadowElement(WebElement parent,String selector) : use this if you want to find all elements from parent DOM

boolean isVisible(WebElement element) : use this if you want to find visibility of element

boolean isChecked(WebElement element) : use this if you want to check if checkbox is selected

boolean isDisabled(WebElement element) : use this if you want to check if element is disabled

String getAttribute(WebElement element,String attribute) : use this if you want to get attribute like aria-selected and other custom attributes of elements.

void selectCheckbox(String label) : use this to select checkbox element using label.

void selectCheckbox(WebElement parentElement, String label) : use this to select checkbox element using label.

void selectRadio(String label) : use this to select radio element using label.

void selectRadio(WebElement parentElement, String label) : use this to select radio element from parent DOM using label.

void selectDropdown(String label) : use this to select dropdown list item using label (use this if only one dropdown is present or loaded on UI).

void selectDropdown(WebElement parentElement, String label) : use this to select dropdown list item from parent DOM using label.

How to use this plugin: You will have to dependency in your project.

Maven

<dependency>
  <groupId>io.github.sukgu</groupId>
  <artifactId>automation</artifactId>
  <version>0.0.4</version>
<dependency>

for html tag that resides under a shadow-root dom element

<properties-page id="settingsPage"> 
  <textarea id="textarea">
</properties-page>

You can use this code in your framework to grab the textarea element Object.

import io.github.sukgu.*;
Shadow shadow = new Shadow(driver);
WebElement element = shadow.findElement("properties-page#settingsPage>textarea#textarea");
String text = element.getText();
Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
Garima
  • 132
  • 3
  • Welcome to StackOverflow! Thank you for your detailed response. Please support it with references. We have guidelines on [How to write a good answer](https://stackoverflow.com/help/how-to-answer). – Joseph Cho May 15 '19 at 13:39
10

With Selenium 4 there is now WebElement.getShadowRoot(). For example:

driver.findElement(By.id("parentId")).getShadowRoot().findElement(By.cssSelector("label")).findElement(By.tagName("input"))

As normal with a #shadow-root, the navigation choices for the next hop are limited. E.g. against Chrome By.cssSelector() and By.className() are valid, but By.id() and By.tagName() fail with org.openqa.selenium.InvalidArgumentException: invalid argument: invalid locator

Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
df778899
  • 10,703
  • 1
  • 24
  • 36
  • 1
    By.xpath() also fails with `org.openqa.selenium.InvalidArgumentException: invalid argument: invalid locator` – Easty77 Nov 14 '22 at 15:47
8

Steps to find out shadow DOM elements using JSExecutor and CSS:

  1. Find out base element i.e parent element of Shadow root element.

  2. Get Shadow root of that element.

  3. And Find your Element on that shadow-root webelement

    example:

<div id="example">
#shadow-root
<div id="root" part="root">
   <div id="label" part="label">ShadowRootLabel</div>
</div>
</ptcs-label>

#Method to find out Shadow Root Element

public WebElement getShadowRootElement(WebElement element) {
WebElement ele = (WebElement) ((JavascriptExecutor)driver)
    .executeScript("return arguments[0].shadowRoot", element);
        return ele;
    }

#Step1 for Example i.e find Base Element:

WebElement root1 = driver.findElement(By.id("example"));

#Step2

//Get shadow root element
WebElement shadowRoot1 = getShadowRootElement(root1);

#Step3 - We need to find elements using CSS Selector which are inside shadow root, xpath will not work here

//Here we will get Element inside Shadow Dom Element
WebElement shadowElement = shadowRoot3.findElement(By.cssSelector("div[id=label]"));
Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
  • It looks like this does not work with Chrome 96+ anymore. When calling `driver.executeScript("return arguments[0].shadowRoot", element)` it will return a com.google.common.collect.Maps$TransformedEntriesMap which cannot be cast to a WebElement. Any Ideas how to fix this? – Jörg Rech Dec 01 '21 at 22:08
  • You need to cast to `SearchContext` or use the new `getShadowRoot()` method: https://titusfortner.com/2021/11/22/shadow-dom-selenium.html – titusfortner Dec 13 '21 at 05:32
6

To demonstrate automation of shadow DOM using Selenium v3.x, ChromeDriver v2.46 and Chrome v73.x here are a couple of approaches which opens the url chrome://downloads/ and using the executeScript() method sends the character sequence pdf as the search text within the Search Box.


Using document.querySelector()

As a canonical approach you can use document.querySelector() method as follows:

  • Code Block:

      import org.openqa.selenium.JavascriptExecutor;
      import org.openqa.selenium.WebDriver;
      import org.openqa.selenium.WebElement;
      import org.openqa.selenium.chrome.ChromeDriver;
      import org.openqa.selenium.chrome.ChromeOptions;
    
      public class shadow_DOM_search_download_querySelector {
    
          public static void main(String[] args)
          {
              System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
              ChromeOptions options = new ChromeOptions();
              options.addArguments("start-maximized");
              options.addArguments("disable-infobars");
              options.addArguments("--disable-extensions"); 
              WebDriver driver = new ChromeDriver(options);
              driver.get("chrome://downloads/");
              JavascriptExecutor jse = (JavascriptExecutor) driver; 
              WebElement search_box = (WebElement) jse.executeScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('downloads-toolbar#toolbar').shadowRoot.querySelector('cr-toolbar#toolbar').shadowRoot.querySelector('cr-toolbar-search-field#search').shadowRoot.querySelector('div#searchTerm input#searchInput')");
              String js = "arguments[0].setAttribute('value','pdf')";
              ((JavascriptExecutor) driver).executeScript(js, search_box);
          }
      }
    

The same solution can be re-written in a step wise fashion as follows:

  • Code Block:

      import org.openqa.selenium.By;
      import org.openqa.selenium.JavascriptExecutor;
      import org.openqa.selenium.WebDriver;
      import org.openqa.selenium.WebElement;
      import org.openqa.selenium.chrome.ChromeDriver;
      import org.openqa.selenium.chrome.ChromeOptions;
    
      public class shadow_DOM {
    
          static WebDriver driver;
          public static void main(String[] args) 
          {   
              System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
              ChromeOptions options = new ChromeOptions();
              options.addArguments("start-maximized");
              //options.addArguments("disable-infobars");
              options.addArguments("--disable-extensions"); 
              driver = new ChromeDriver(options);
              driver.get("chrome://downloads/");
              WebElement root1 = driver.findElement(By.tagName("downloads-manager"));
              WebElement shadow_root1 = expand_shadow_element(root1);
    
              WebElement root2 = shadow_root1.findElement(By.cssSelector("downloads-toolbar#toolbar"));
              WebElement shadow_root2 = expand_shadow_element(root2);
    
              WebElement root3 = shadow_root2.findElement(By.cssSelector("cr-toolbar#toolbar"));
              WebElement shadow_root3 = expand_shadow_element(root3);
    
              WebElement root4 = shadow_root3.findElement(By.cssSelector("cr-toolbar-search-field#search"));
              WebElement shadow_root4 = expand_shadow_element(root4);
    
              WebElement search_term = shadow_root4.findElement(By.cssSelector("div#searchTerm input#searchInput"));
              String js = "arguments[0].setAttribute('value','pdf')";
              ((JavascriptExecutor) driver).executeScript(js, search_term);
    
              WebElement search_button = shadow_root4.findElement(By.cssSelector("paper-icon-button#icon"));
              search_button.click();
    
              System.out.println("Search Button Clicked");
          }
    
          public static WebElement expand_shadow_element(WebElement element)
          {
              WebElement shadow_root = (WebElement)((JavascriptExecutor)driver).executeScript("return arguments[0].shadowRoot", element);
              return shadow_root;
          }
    
      }
    

  • Console Output:
Search Button Clicked

  • Browser Snapshot:

shadowDOM


Outro

As per the discussion in Determine the fate of experimental '>>>' combinator the >>> combinator, which was the replacement for /deep/ combinator for piercing all the shadow DOM boundaries to style, which was implemented behind the flag in Blink is deprecated.

Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
5

Shadow DOM in Chrome v96 (and above) and Selenium

With the availability of Chrome v96, has made its shadow root return values compliant with the W3C WebDriver specification.

As @titusfortner mentions in their comment:

Now that Chrome supports shadow root via the driver, shadowRoot JS calls are returned per the spec with a shadow root element key (shadow-6066-11e4-a52e-4f735466cecf). Selenium 4 has a new ShadowRoot class to support this, but we didn't include the translation code that we do when execute script calls return elements. This has been fixed and will be available in Selenium 4.1.

The only difference is that you'll need to cast to ShadowRoot instead of WebElement.

So moving forward using Selenium with Microsoft Edge and Google Chrome v96 and greater, we need to use the new shadow root method as follows:

  • Java example:

    driver.get("http://watir.com/examples/shadow_dom.html");
    WebElement shadowHost = driver.findElement(By.cssSelector("#shadow_host"));
    SearchContext shadowRoot = shadowHost.getShadowRoot();
    WebElement shadowContent = shadowRoot.findElement(By.cssSelector("#shadow_content"));
    Assertions.assertEquals("some text", shadowContent.getText());
    
  • Python example:

    driver.get('http://watir.com/examples/shadow_dom.html')
    shadow_host = driver.find_element(By.CSS_SELECTOR, '#shadow_host')
    shadow_root = shadow_host.shadow_root
    shadow_content = shadow_root.find_element(By.CSS_SELECTOR, '#shadow_content')
    assert shadow_content.text == 'some text'
    
  • C# example:

    _driver.Navigate().GoToUrl("http://watir.com/examples/shadow_dom.html");
    var shadowHost = _driver.FindElement(By.CssSelector("#shadow_host"));
    var shadowRoot = shadowHost.GetShadowRoot();
    var shadowContent = shadowRoot.FindElement(By.CssSelector("#shadow_content"));
    
  • Ruby example:

    @driver.get('http://watir.com/examples/shadow_dom.html')
    shadow_host = @driver.find_element(css: '#shadow_host')
    shadow_root = shadow_host.shadow_root
    shadow_content = shadow_root.find_element(css: '#shadow_content')
    expect(shadow_content.text).to eq 'some text'
    

tl; dr

Read more at:

Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
0

I can access of multi-level of shadow-root DOM elements using this recursive function.

private getHTMLElementWithShadowRoot(htmlElement: HTMLElement, nameHtmlElements: string[]): HTMLElement {
            if (nameHtmlElements.length === 0) return htmlElement;
            return this.getHTMLElementWithShadowRoot(
                htmlElement.shadowRoot ?
                    htmlElement.shadowRoot.querySelector(nameHtmlElements[0]) :
                    htmlElement.querySelector(nameHtmlElements[0]), nameHtmlElements.slice(1)
            );
 }

I use in this way:

let appComponent = document.body.querySelector("app-component") as HTMLElement;
let appHeader = this.getHTMLElementWithShadowRoot(appComponent, ["main-component", "header-component", "#app-header"]);

Here you can see the DOM enter image description here

You have to write the underline names in the array string nameHtmlElements without the first name, since the first name is passed in the first parameter of the function.