0

Here's my script. I have a .csv file that contain URL adress. For each URL, i'm scrapping text in the <div> element with the class "sc-kIKDeO.hIofms". But, when I run my script, it opens a new browser windows for each URL, which is not really effective & quick. I need to open a tab in the same browser instance for each URL. I tried differents things but I still don't manage to do this... If someone could help me please?

const puppeteer = require("puppeteer");
const fs = require("fs");
const urlList = fs.readFileSync("sample.csv", "utf-8").split("\n");

urlList.forEach(async (url) => {
  const browser = await puppeteer.launch({
    executablePath:
      "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
    headless: true,
    arg: ["--incognito"],
  });
  const page = await browser.newPage();
  await page.setRequestInterception(true);
  page.on("request", (request) => {
    if (
      request.resourceType() === "image" ||
      request.resourceType() === "stylesheet" ||
      request.resourceType() ===
        "font" /*|| request.resourceType() === 'script' && !request.url().includes('.sc-kIKDeO.hIofms')*/
    ) {
      request.abort();
    } else {
      request.continue();
    }
  });
  await page.goto(url, { waitUntil: "networkidle0", timeout: 60000 });
  try {
    await page.waitForSelector(".sc-kIKDeO.hIofms");
    const text = await page.evaluate(
      () => document.querySelector(".sc-kIKDeO.hIofms").textContent
    );
    console.log(text);
  } catch (error) {
    console.log("error:${error}");
  }
  await browser.close();
});

I'm trying to scrape data from differents URL by opening each URL in differents tab of the same browser's instance.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
  • Please see the dupes as well as [Why is "Can someone help me?" not an actual question?](https://meta.stackoverflow.com/questions/284236/why-is-can-someone-help-me-not-an-actual-question). I suggest using [prettier](https://prettier.io/playground/) to format your code in the future so it's comprehensible (I did it for you this time). Thanks. – ggorlen Jan 24 '23 at 23:27

0 Answers0