How to return a value (object) from inside a puppeteer async function?

Question

I am using the puppeteer library to do some web scraping. I want to a return an async object to the outside function which calls it, so that I can insert it into my db.

The problem is, in order to dispose of the browser object that the async function is operating on, you must call "await browser.close();" as the last call.

The other problem is, console logging the result of my function only displays a promise.

I've tried putting my return statement after my await browser.close() method, as well as putting "return await mainObj" but it still returns a promise.

const puppeteer = require('puppeteer');

async function webScraper(u, p, url) {
    const browser = await puppeteer.launch();

    const page = await browser.newPage();



    await page.goto(url);
    await page.waitForSelector('#UserName')
    await page.focus('#UserName')
    await page.keyboard.type(u)
    await page.waitForSelector('#Password')
    await page.focus('#Password')
    await page.keyboard.type(p)

    // Code edited out to keep private what website I'm using.
    // Here it loops through page contents and constructs arrays which are used to construct my mainObj.

    let mainObj = {};
    let secondObj = {};

    for (i = 0; i < descArray.length; i++) {

        secondObj[descArray[i]] = [ammtArray[i], datesArray[i]]

    }
    secondObj[totaldescArray[0]] = totalammtArray[0]
    mainObj[datesArray[0]] = secondObj


    console.log(mainObj, 'here')

    await browser.close();

    return await mainObj

}
console.log(webScraper("username", "password", "url"))

Console.logging the mainObj in the function returns my expected object. But console.logging the result of calling the function webScraper() returns a Promise. Whether or not I use "return await mainObj" or "return mainObj" (keep in mind the object is awaited because the omitted part that constructs the arrays is async).

Could somebody please point out what I'm doing wrong? Thank you.

score 2 · Accepted Answer · answered Aug 20 '19 at 14:35

2

Perhaps you should wait for promise returned from webScraper to finish processing. Since you cannot use await outside of async function, use Promise.then:

webScraper("username", "password", "url")
     .then(mainObj => console.log(mainObj));

answered Aug 20 '19 at 14:35

Nenad

24,809
11
75
93

1

This worked, I figured it was something to do with the way I was calling and receiving the result. I'm still fairly new to async.... so thank you very much for the knowledge. I'll mark this is answered ASAP (and give you your 10k reputation!) – Will Aug 20 '19 at 14:39
10k it is! Thank you @Will! :) – Nenad Aug 20 '19 at 16:01

How to return a value (object) from inside a puppeteer async function?

1 Answers1