Trying to scrape Apple review with puppeteer, but it returns empty

Question

I'm trying to scrape all the reviews below;

https://apps.apple.com/jp/app/mcdonalds-japan/id413618155?l=en#see-all/reviews

If you could see the reviews are occluded, so you need;

Click to see the review
Scroll to see more

I'm still far to go, but I've tried this code;

import puppeteer from 'puppeteer';

const url = "https://apps.apple.com/jp/app/mcdonalds-japan/id413618155?l=en#see-all/reviews";

(async () => {

  try {

    const browser = await puppeteer.launch({
      headless: false
    });
    const page = await browser.newPage();

    await page.goto(
      url,{
      waitUntil: "domcontentloaded"
    });
    await page.waitForTimeout(4000);
    let result = await page.evaluate(() => {
      const items = Array.from(document.querySelectorAll("div"));
      return items
    })

    console.log(result);
    await browser.close();


  } catch (e) {
    console.error();
  }

})();

But the result returns empty, not sure what I did wrong.

[
  {}, {}, {}, {}, {}, {}, {}, {},
  {}, {}, {}, {}, {}, {}, {}, {},
  {}, {}, {}, {}, {}, {}, {}, {},
  {}, {}, {}
]

Does this answer your question? [Puppeteer page.evaluate querySelectorAll return empty objects](https://stackoverflow.com/questions/46377955/puppeteer-page-evaluate-queryselectorall-return-empty-objects) — ggorlen, Jul 12 '21 at 17:02

score 0 · Answer 1 · answered Jul 12 '21 at 14:27

Unfortunately, page.evaluate() can only transfer serializable values (roughly, the values JSON can handle). As document.querySelectorAll() returns a collection of DOM elements that are not serializable (they contain methods and circular references), each element in the collection is replaced with an empty object. You need to return either serializable value (for example, an array of texts or attributes) or use something like page.$$(selector) and ElementHandle API.

Trying to scrape Apple review with puppeteer, but it returns empty

1 Answers1