puppeteer, nodejs. I'm unsure why trying to pass values through a variable gives an error "is undefined", but when I use a normal int it works

Question

I'm new to all this so any suggestions about resources to learn from would also be appreciated! So as I mentioned I'm using puppeteer, js and node. When I try to run the blow code I get "placeholder is not defined" but if I just put in the number 4 I get the link I'm looking for which in this example is https://stackoverflow.com/users.

I'm really unsure why this happened, one of the closest examples of this error suggested that it loads the variable and runs the code before the page loads, but I don't think that's the issue.

Essentially I want to get a list of all the links on a page or more specifically in lists and access them, I'd bet this isn't the best way to navigate a site or links but have been trouble finding some resources that are useful.

const puppeteer = require('puppeteer');
const fs = require('fs');

async function MyFunction(scrapelink){

    const browser = await puppeteer.launch({headless:false});
    const page = await browser.newPage();

    await page.goto(scrapelink, {waitUntil: "networkidle2"});

    const placeholder = 4;

    const fourthLink = await page.evaluate(() => document.links[placeholder].href);
    //changing placeholder with just 4 works.
    console.log(fourthLink);

    browser.close();
}

MyFunction("https://stackoverflow.com/");

Btw, `console.log(fourthLink);` is logging a promise. Are you sure you don't want `fourthLink.then(x => console.log(x));` — Keynan, Jan 27 '20 at 22:37
I probably do. But if the only main issue that causes is being terribly inefficient code and using more than necessary system resources that isn't the biggest concern, I will update and be more conscious of that in the future! I do get an error "fourthLink.then is not a function" though — magnuson, Jan 28 '20 at 04:06

score 0 · Accepted Answer · answered Jan 27 '20 at 22:35

0

You must pass in your variable like so

const fourthLink = await page.evaluate(placeholder => {
    return document.links[placeholder].href
}, placeholder);

See this question about passing variables to .evaluate()

answered Jan 27 '20 at 22:35

Nermin

749
7
17

Thank you so much! Both these answer were extremely helpful! – magnuson Jan 28 '20 at 04:02
You're welcome! Please feel free to accept any answer you consider has answered your question the best :) – Nermin Jan 28 '20 at 08:17

score 0 · Answer 2 · answered Jan 27 '20 at 23:09

Please read the documentation here and your code should work as follows:

    const puppeteer = require('puppeteer');

    async function MyFunction(scrapelink) {

    const browser = await puppeteer.launch({ headless: false });
    const page = await browser.newPage();

    await page.goto(scrapelink, { waitUntil: "networkidle2" });

    const placeholder = 4;

    // you need to pass the placeholder as follows 
    const fourthLink = await page.evaluate(link_position => {
        return document.links[link_position].href
    }, placeholder);
    console.log(fourthLink);

    browser.close();
}

MyFunction("https://stackoverflow.com/");

puppeteer, nodejs. I'm unsure why trying to pass values through a variable gives an error "is undefined", but when I use a normal int it works

2 Answers2