Here's my script. I have a .csv file that contain URL adress. For each URL, i'm scrapping text in the <div>
element with the class "sc-kIKDeO.hIofms"
. But, when I run my script, it opens a new browser windows for each URL, which is not really effective & quick. I need to open a tab in the same browser instance for each URL. I tried differents things but I still don't manage to do this... If someone could help me please?
const puppeteer = require("puppeteer");
const fs = require("fs");
const urlList = fs.readFileSync("sample.csv", "utf-8").split("\n");
urlList.forEach(async (url) => {
const browser = await puppeteer.launch({
executablePath:
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
headless: true,
arg: ["--incognito"],
});
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on("request", (request) => {
if (
request.resourceType() === "image" ||
request.resourceType() === "stylesheet" ||
request.resourceType() ===
"font" /*|| request.resourceType() === 'script' && !request.url().includes('.sc-kIKDeO.hIofms')*/
) {
request.abort();
} else {
request.continue();
}
});
await page.goto(url, { waitUntil: "networkidle0", timeout: 60000 });
try {
await page.waitForSelector(".sc-kIKDeO.hIofms");
const text = await page.evaluate(
() => document.querySelector(".sc-kIKDeO.hIofms").textContent
);
console.log(text);
} catch (error) {
console.log("error:${error}");
}
await browser.close();
});
I'm trying to scrape data from differents URL by opening each URL in differents tab of the same browser's instance.