Is there any way to get element info inside the shadow root with Selenium?

Question

I am trying to scrape some info from a website with java and selenium. However, because of the shadow-root I cannot reach any web element. When i try to get html, it returns empty array. Is there any way to reach the info inside shadow-root or is it impossible? html structure of the website

Answer 1

You can try JavascriptExecutor to find elements within shadow-root.

WebElement rootElement = (WebElement) ((JavascriptExecutor)driver).executeScript("return arguments[0].shadowRoot", PreRootElement);

For details please refer: http://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver

Answer 2

Mavni's answer solved the problem for me, but as a refinement I wanted to be able to use a By as a locator (e.g. to wait on an ExpectedCondition).

public class ByShadowRoot extends By {

    @Override
    public WebElement findElement( SearchContext context ) {

        RemoteWebElement element = ( RemoteWebElement ) context;
        JavascriptExecutor driver = ( JavascriptExecutor ) element.getWrappedDriver();
        return ( WebElement ) driver.executeScript( "return arguments[0].shadowRoot" , element );
    }

    @Override
    public List<WebElement> findElements( SearchContext context ) {

        WebElement element = findElement( context );
        return element != null ? Collections.singletonList( element ) : Collections.emptyList();
    }

    @Override
    public String toString() {

        return "By#shadow-root";
    }
}

This was then used like this (this example was used to wait for a ComboBox to pop-up in Vaadin Flow):

ByShadowRoot byShadowRoot = new ByShadowRoot();
By comboBoxItem = new ByChained( By.id( "overlay" ) , byShadowRoot , By.id( "content" ) , byShadowRoot , By.id( "selector" ) ,
    By.tagName( "vaadin-combo-box-item" ) );

(ByChained is in the selenium-support.jar)

A subtlety there was that By.tagName() failed if it was immediately after the byShadowRoot, with WebDriverException: unknown error: b.getElementsByTagName is not a function. I think this because the #shadow-root is just a document fragment in the DOM, as opposed to a document element.