Best way to run a Puppeteer function in hourly intervals

Question

I have a simple Puppeteer script that I want to run in an hourly interval. I've taken two approaches 1) using setIntervals 2) using a sleep function. I know there maybe other approaches such as cron jobs and would like to ask what would be the best way to solve this.

Use set intervals

const runPuppeteer = () => {
    puppeteer.launch({
        headless: isHeadless,
        args: [
            '--no-sandbox',
            '--disable-setuid-sandbox'
        ]
    }).then(async browser => { 
       //Do something... 

       await browser.close()  // close browser after
})}

setInterval(runPuppeteer, 60 * 1000 * 60);

Use sleep

const runPuppeteer = () => {
    puppeteer.launch({
        headless: isHeadless,
        args: [
            '--no-sandbox',
            '--disable-setuid-sandbox'
        ]
    }).then(async browser => { 

       while(true){
          //Do something...           
          sleep.sleep(60 * 60)
       }
})}

For some reason when I took 2) sleep approach and looked at htop there were multiple threads and tasks (I think this is mostly due to my code..)

htop image

Sorry, but doesn't setTimeout execute only once instead of intervals? — refinish_devloper, Jun 24 '20 at 09:37
you can also use service like AWS lambda + Cloudwatch event to run your tasks — Chau Giang, Jun 24 '20 at 09:45
recursive ```setTimeout``` can be used. https://stackoverflow.com/questions/7188145/call-a-javascript-function-every-5-seconds-continuously — xMayank, Jun 24 '20 at 09:49

Edi Imanto · Answer 1 · 2020-06-25T06:16:57.457

Maybe this will help you design a more sophisticated cron jobs

const cron = require('node-cron')
const execute = require('child_process').exec
const puppeteer = require('puppeteer')
const processIDs = []

const createBrowser = async () => {
    const browser = await puppeteer.launch()
    processIDs.push(browser.process().pid)
    return browser
}

const closeBrowser = async (browser) => {
    await browser.close()
    for (let i = 0; i < processIDs.length; i++) {
        execute(`echo ${process.env.PASSWD} | sudo -S kill -9 ${processIDs[i]}`)
    }
}

const runHourly = async () => {
    const browser = await createBrowser()
    const [page] = await browser.pages()
    const response = await page.goto(url)
    const screenshot = await page.screenshot({
        path: 'screenshot_' + Date.now() + '.png'
    })
    const closeTab = await page.close()
    const exit = await closeBrowser(browser)
}

cron.schedule('0 * * * *', () => {
    console.log('running every minute 0 / hourly')
    await runHourly()
})

Best way to run a Puppeteer function in hourly intervals

1 Answers1