Puppeteer $.eval selecting nested elements

Question

Lets say i'm give situation like this page

<div id="details-container" class="style-scope ytd-channel-about-metadata-renderer">
         <yt-formatted-string class="subheadline style-scope ytd-channel-about-metadata-renderer">Details</yt-formatted-string>
        <table class="style-scope ytd-channel-about-metadata-renderer">
          <tbody class="style-scope ytd-channel-about-metadata-renderer"><tr class="style-scope ytd-channel-about-metadata-renderer">
            <td class="label style-scope ytd-channel-about-metadata-renderer">
              <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer"></yt-formatted-string>
            </td>
            <td class="style-scope ytd-channel-about-metadata-renderer">
              <ytd-button-renderer align-by-text="" class="style-scope ytd-channel-about-metadata-renderer" button-renderer=""></ytd-button-renderer>
              <div id="captcha-container" class="style-scope ytd-channel-about-metadata-renderer"></div>
              <div id="email-container" class="style-scope ytd-channel-about-metadata-renderer"></div>
              <a id="email" target="_blank" class="style-scope ytd-channel-about-metadata-renderer" href="mailto:undefined" hidden=""></a>
            </td>
          </tr>
          <tr class="style-scope ytd-channel-about-metadata-renderer">
            <td class="label style-scope ytd-channel-about-metadata-renderer">
              <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer"><span class="deemphasize style-scope yt-formatted-string"> Location:   </span></yt-formatted-string>
            </td>
            <td class="style-scope ytd-channel-about-metadata-renderer">
              <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer">YourCountry</yt-formatted-string>
            </td>
          </tr>
        </tbody></table>
      </div>

Lets say i need get the "YourCountry" how do i actually get to this element?

So far i tried:

  const location = await page.$$eval(
    "#details-container > table > tbody:nth-child(1) > tr:nth-child(1) > yt-formatted-string",
    locationEl => locationEl.innerHTML
  );
console.log(location) // Undefined

Not sure how to go about it, trying to return just tr and then again evaluating tr[1] doesn't work as it says that tr doesn't have a function .$$eval.

Note that i'm using apify to get the page.

score 4 · Accepted Answer · answered Mar 03 '19 at 15:28

In the HTML you’ve provided, the yt-formatted-string element you want is a direct child of the second td under the second tr, but you try to match it a yt-formatted-string that’s a direct child of the second tr. You need to fix your selector. For example:

console.log("HTML:", document.querySelector("#details-container > table > tbody > tr:nth-child(2) > td:nth-child(2) > yt-formatted-string").innerHTML)

<div id="details-container" class="style-scope ytd-channel-about-metadata-renderer">
  <yt-formatted-string class="subheadline style-scope ytd-channel-about-metadata-renderer">Details</yt-formatted-string>
  <table class="style-scope ytd-channel-about-metadata-renderer">
    <tbody class="style-scope ytd-channel-about-metadata-renderer">
      <tr class="style-scope ytd-channel-about-metadata-renderer">
        <td class="label style-scope ytd-channel-about-metadata-renderer">
          <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer"></yt-formatted-string>
        </td>
        <td class="style-scope ytd-channel-about-metadata-renderer">
          <ytd-button-renderer align-by-text="" class="style-scope ytd-channel-about-metadata-renderer" button-renderer=""></ytd-button-renderer>
          <div id="captcha-container" class="style-scope ytd-channel-about-metadata-renderer"></div>
          <div id="email-container" class="style-scope ytd-channel-about-metadata-renderer"></div>
          <a id="email" target="_blank" class="style-scope ytd-channel-about-metadata-renderer" href="mailto:undefined" hidden=""></a>
        </td>
      </tr>
      <tr class="style-scope ytd-channel-about-metadata-renderer">
        <td class="label style-scope ytd-channel-about-metadata-renderer">
          <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer"><span class="deemphasize style-scope yt-formatted-string"> Location:   </span></yt-formatted-string>
        </td>
        <td class="style-scope ytd-channel-about-metadata-renderer">
          <yt-formatted-string class="style-scope ytd-channel-about-metadata-renderer">YourCountry</yt-formatted-string>
        </td>
      </tr>
    </tbody>
  </table>
</div>

And you should be able to call $$eval &c. if you have an ElementHandle. The problem is that your selector didn’t match, so you didn’t have one.

score 2 · Answer 2 · answered Mar 04 '19 at 11:57

I prefer to use jQuery. It is the best way how to query elements. You can inject jQuery for example from Apify utils.

const { puppeteer } = Apify.utils;

await puppeteer.injectJQuery(page);
const location = await page. evaluate(() => {
  return $('#details-container yt-formatted-string').last().text();
});
console.log(location);

Puppeteer $.eval selecting nested elements

2 Answers2