Can't locate and click on a terms of conditions button

Question

I am new to pupeeteer and first what i am trying to do is loading a page and clicking on a button. However, it can't locate the element. I assume this is because I need to locate the parent or parent's parent element.

<button role="button" data-testid="uc-accept-all-button" class="sc-gsDKAQ fHGlTM" style="border: 2px solid rgb(247, 196, 0); padding: 0.375rem 1.125rem; margin: 0px 6px;">Accept All</button>

This is the full css selector taken from inspect

#focus-lock-id > div.sc-furwcr.lhriHG > div > 
div.sc-bYoBSM.egarKh > div > div > div.sc-dlVxhl.bEDIID > 
div > button:nth-child(3)

Here's my code:

const puppeteer = require("puppeteer");

async function launch() {
  const browser = await puppeteer.launch({
    headless: false,
    defaultViewport: false,
  });
  const page = await browser.newPage();

  await page
    .goto("", {
      waitUntil: "networkidle0",
    })
    .catch((err) => console.log("error loading url", err));
  page.click('button[data-testid="uc-deny-all-button"]');
}
launch();

It's a simple accept and conditions block where I would want to click on an "Accept all" button to close it and proceed further. What usual actions do I need to wait for the parent element first then dig further? Or there is an easy way?

This is the website I am trying to close terms and conditions for: https://www.partslink24.com/

ggorlen · Accepted Answer · 2023-04-25T18:38:54.047

A few problems:

The selector appears dynamically after the page has loaded, so you should waitForSelector rather than assuming networkidle0 will be enough to catch the button.
The selector you want is in a shadow root, so select the root and dip into it with .shadowRoot.
Your .click() call must be awaited.

const puppeteer = require("puppeteer"); // ^19.7.2

let browser;
(async () => {
  browser = await puppeteer.launch({headless: true});
  const [page] = await browser.pages();
  const url = "https://www.partslink24.com/";
  await page.goto(url, {waitUntil: "domcontentloaded"});
  const rootSel = "#usercentrics-root";
  const btnSel = 'button[data-testid="uc-deny-all-button"]';
  const rootContainer = await page.waitForSelector(rootSel);
  const root = await rootContainer.evaluateHandle(el => el.shadowRoot);
  const btn = await root.waitForSelector(btnSel);
  await btn.click();
  await page.waitForFunction(`
    !document.querySelector("${rootSel}")
      .shadowRoot
      .querySelector('${btnSel}')
  `);
  await page.screenshot({path: "test.png", fullPage: true});
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

An even shorter way is to use pierce/ to automatically query within open shadow roots:

// ...
  const btn = await page.waitForSelector("pierce/ " + btnSel);
  await btn.click();
// ...

See also Popup form visible, but html code missing in Puppeteer which has the same #usercentrics-root and offers other suggestions for getting rid of the popup.

Since the original post, the site has changed, as has the Puppeteer shadow DOM API.

The new interface is confusing, arguably using a dark pattern. There's a toggle to tick to not sell your personal information, but it's not clear which side corresponds to which setting. I assume ticking it on opts out.

From a Puppeteer perspective, there's now >>> which traverses the shadow DOM, replacing pierce/.

const puppeteer = require("puppeteer"); // ^19.11.1

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const url = "https://www.partslink24.com/";
  await page.goto(url, {waitUntil: "domcontentloaded"});
  const rootSel = "#usercentrics-root";
  const toggleSel = "#toggle-toggle-ccpa-banner";
  const btnSel = '[data-testid="uc-ccpa-button"]';
  const toggle = await page.waitForSelector(">>> " + toggleSel);
  await toggle.click();
  const btn = await page.waitForSelector(">>> " + btnSel);
  await btn.click();
  await page.waitForFunction(`
    !document.querySelector("${rootSel}")
      .shadowRoot
      .querySelector('${btnSel}')
  `);
  await page.screenshot({path: "test.png", fullPage: true});
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

Thank you so much May i ask you why we are doing waitForFunctions() what do we say when we call it , also evaluateHandle not very clear. thank you so much man — bobby bobby, Oct 13 '22 at 14:33
`waitForFunction` is used to delay the screenshot until the button disappears. Maybe unnecessary but I like to make sure there's no race condition. You can probably skip these lines. They're there to show that the code works. `evaluateHandle` returns an ElementHandle for the shadow root back to Node context so I can wait for the button to appear within it. — ggorlen, Oct 13 '22 at 14:41
Thank you man, what about inputs like there is login password form on the right is there are need to wait for the agree with terms div disappear before actually locating the inputs? await page.type('#username', 'username'); await page.type('#password', 'password'); await page.click('#submit'); await page.waitForNavigation(); — bobby bobby, Oct 13 '22 at 14:48
If there's any uncertainty about whether the element exists or not, I wait. You should use the [`Promise.all` pattern](https://stackoverflow.com/questions/50074799/how-to-login-in-puppeteer/50078167#50078167) when triggering navigation. — ggorlen, Oct 13 '22 at 14:54
Thank you man and i promise last one :) i edited my question, wanted to know how to find a specific element in a bunch of divs , can i simply say i want brand logo class where title is Ford — bobby bobby, Oct 13 '22 at 15:13
Please ask a new question if you have a new question. The point of the site is to be a resource for future visitors with one specific problem per thread. I rolled back your edit. Thanks. If you want a specific div, please specify what makes it special--why do you want that div? If the title is Ford, then select based on that. — ggorlen, Oct 13 '22 at 15:14

Can't locate and click on a terms of conditions button

1 Answers1

Linked