How to get text from shadow root element?

Question

I have a web page that I am automating using Puppeteer. I want to get a text out of shadow root element. My confusion is that shadow root element visibility is enabled by enabling property in the browser setting. But when the script run a new browser instance is launched with shadow root disabled by default. So how can I access the element. Is there any way to enabled shadow root pro grammatically. I am attaching screenshot of my DOM element.

Answer 1

Since this PR https://github.com/puppeteer/puppeteer/pull/6509 you can use the builtin pierce query handler. In the HTML example that comes with the docs:

<div>
  <custom-element>
    <div></div>
  </custom-element>
</div>

This call gives you access to the div in the shadow root:

await page.$$('pierce/div')

The query-selector-shadow-dom npm package proposed by @theDavidBarton shouldn't be needed anymore.

Answer 2

You can use query-selector-shadow-dom npm package for this purpose with puppeteer.

const puppeteer = require('puppeteer');
const {  QueryHandler } = require('query-selector-shadow-dom/plugins/puppeteer');
(async () => {
    try {
        await puppeteer.__experimental_registerCustomQueryHandler('shadow', QueryHandler);
        const browser = await puppeteer.launch({
            headless: false
        });
        const page = await browser.newPage();
        await page.goto('http://some-site.com/');

        await page.waitForSelector('shadow/div'); // if it'd have a class you could replace "div" with the ".class-name"
        const shadowDiv = await page.$('shadow/div');
        const text = await page.evaluate(el => el.innerText, shadowDiv);

        await browser.close();
    } catch (e) {
      console.error(e);
    }
})()

---

You mentioned you need to apply shadow DOM in chrome settings, I suggest to create a chrome profile (Custom Profile 1) with the required settings and then use this profile with puppeteer:

E.g.:

const browser = await puppeteer.launch({
    headless: false,
    args: [
      '--user-data-dir=C:\\Users\\user.name\\AppData\\Local\\Chromium\\User Data',
      '--profile-directory=Custom Profile 1'
    ]
  })

The example above is for Windows, see more platforms and more info here.