0

I have a puppeteer script which takes a URL and scrapes it.

Every time that my website is visited, it loads up a new browser process and runs:

const browser = await puppeteer.launch();
const page = await browser.newPage();

await page.goto(url,{
    waitUntil: 'networkidle2'
});

await browser.close();
res.end(data);

Instead of having to load a new browser process on each load, how can I have one browser running 24/7 that can still receive the URL parameter via a GET request?

jefiwo
  • 111
  • 1
  • 1
  • 6
  • Are you using Express? What's `res.end(data)`? Are you looking for something like [this](https://stackoverflow.com/a/67910262/6243352)? Basically move the `launch` outside of the route. What's your use case here? Thanks. – ggorlen Feb 14 '22 at 02:30
  • I just want to say that having a headless browser running 24/7 could eat up your resources/make things run slower. I would recommend checking out Apify, where you can build an API based on a scraper in Cheerio/Puppeteer/Playwright, then asynchronously call that API whenever you want within your main project. – mstephen19 Feb 14 '22 at 02:53

0 Answers0