How do you pass variables and function inside of Puppeter?

Question

I need to create a puppeteer script that will be able to send requests as well. I was using something that looks like this below

    async function start() {
    let item = {cin: "1234567"}
    let request = require('request-promise').defaults({followAllRedirects:true})
    async function loadPage(){
     puppeteer.launch({ headless: false, executablePath: getChromiumExecPath() }).then(async browser => {
            const page = (await browser.pages())[0]
            await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36');
            await page.setViewport({width: 1366, height: 768});
            await page.evaluate(() => {
                console.log(document.cookie)
                let cookies = document.cookie
                console.log(item.cin)
                let dataString = `{${item.tcin}}}`
                let atc = request.post({
                    headers : {
                        "accept": "application/json",
                        "accept-language": "en-US,en;q=0.9",
                        "content-type": "application/json",
                        "sec-fetch-dest": "empty",
                        "sec-fetch-mode": "cors",
                        "sec-fetch-site": "same-site",
                        "x-application-name": "web",
                        "cookie": `${cookies}`
                      },
                      referrer: "RANDOMURLHERE.com",
                    body: dataString
                    
                }).then((res)=>{
                    console.log(res)
                }).catch((err)=>{
                    console.log(err)
                })
                return atc
} 
    
    
    } (item));

})}} 
await loadPage

I get Error: Evaluation failed: ReferenceError: item is not defined. So I tried reading upon it. I ran into this forum Puppeteer: pass variable in .evaluate() So I tried

await page.evaluate((item) => {
                console.log(document.cookie)
                let cookies = document.cookie
                console.log(item)
                 let datastring = `{${item.tcin}}` 
                let atc = request.post({
                    headers : {
                        "accept": "application/json",
                        "accept-language": "en-US,en;q=0.9",
                        "content-type": "application/json",
                        "sec-fetch-dest": "empty",
                        "sec-fetch-mode": "cors",
                        "sec-fetch-site": "same-site",
                        "x-application-name": "web",
                        "cookie": `${cookies}`
                      },
                      referrer: "UNIMPORTANT",
                    body: dataString
                    
                }).then((res)=>{
                    console.log(res)
                }).catch((err)=>{
                    console.log(err)
                })
                return atc

           }, (item.tcin));

It worked and I was able to see the value of item.tcin in the console. Now there was a new problem. Now I get Error: Evaluation failed: ReferenceError: request is not defined. So then I passed in request and then I get the same error with post. There has to be an easier way for me to use all of my variables and functions inside of a page.evaluate. If you know a solution please let me know and if you have any questions to clarify something please let me know. Also I should that the entire page.evaluate and the request inside works when I hardcode the item.tcin but that value changes so I can't hardcode it.

Vaviloff · Answer 1 · 2020-12-20T02:54:01.260

Any code that you write inside of page.evaluate will be executed in a browser (just as if you opened a fresh browser, pressed F12 and pasted your code in Devtools console). This is called the browser context. Does request exist there? No, it's a node.js module!

Everything else that you write in your code is executed not in a browser but in node.js context, including any modules that you import/require.

// This is the node.js context, you can use node modules
const request = require('request');
const puppeteer = require('puppeteer');

async function run() {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  const selector = 'span.some__name';

  await page.goto(url);
  
  const clientName = await page.evaluate(function(selector){
    // This function is basically run inside of devtools console
    return document.querySelector(selector).innerText;
    // End of the code that will be executed in the browser context
  }, selector);

  console.log(clientName);
}

run();
// end of the node.js context

Now there is a theoretical possibility to do what you want: the page.exposeFunction method, with which you can add ability to call this function from page.evaluate. However that will still work in node.js context.

So if you want to make some HTTP requests on behalf of a web page, you should use something like axios by adding it to the page target with page.addScript method.

I am a bit confused on what you mean with using axios and the "browser version" — Prada, Dec 19 '20 at 18:53
Also can you show an example of how you would use the page.exposeFunction in order to get those variables and be able to use fetch. — Prada, Dec 19 '20 at 18:56
The only work around that I found was just passing in the variable and then instead of using the request-promise library I would just use fetch. — Prada, Dec 19 '20 at 20:03
Re "browser version": yes, twas unclear, updated the answer. For a sample see https://stackoverflow.com/a/65367516/2715393 — Vaviloff, Dec 20 '20 at 02:55

How do you pass variables and function inside of Puppeter?

1 Answers1