Problem with Node.js Request module inside my own function

Question

I have this code:

const request = require('request');

let url = "https://api.warframe.market/v1/items";

let options = {json: true};
let item_urls;
//const response;
request(url, options, (error, res, body) => {
    if (error) {
        return  console.log(error)
    };

    if (!error && res.statusCode == 200) {
        console.log(body);
        item_urls = body.payload.items;

    };
});

It works fine and returns the JSON to the variable item_urls.

I then tried to wrap my own function around it like so:

const request = require('request');

let url = "https://api.warframe.market/v1/items";

let options = {json: true};
let item_urls;
//const response;

function getitems(){
request(url, options, (error, res, body) => {
    if (error) {
        return  console.log(error)
    };

    if (!error && res.statusCode == 200) {
        console.log(body);
        item_urls= body.payload.items;

    };
});
};
getitems()

But it doesn't work, item_urls ends up being undefined. The code is identical except for being wrapped inside my function. In case you are wondering why I want to it wrapped inside a function - I need to pass a parameter to it but I've removed this logic for simplicity and to demonstrate the issue.

Thanks,

EDIT: Additional details I forgot to mention sorry.

So I am running this code directly line by line in my IDE and sending the lines to the node REPL so that I can test code line by line. If we look at the first block of code:

I highlight it all and run it. Since request is async, I wait for it to complete first, I know it's completed because the callback function outputs the body to the console, so I wait until I see a load of data get printed to the console, this is how I verify the command completed successfully. Straight after the line of code which outputs the body to the console it then dumps the body into my variable. To confirm this has worked I then type item_urls at the REPL terminal itself and can see the output.

With the second block of code, I open a new REPL so everything is clean. I then execute all the lines of code again and wait... What I am expecting is for the request async call to take a few seconds to complete like the first code block. However, nothing get's outputted to the console. I wait up to 30 secs (the previous block of code only took around 5 secs) then I type item_urls at the REPL terminal again and it says undefined.

I am convinced it has something to do with the async and callback nature but I can't figure out what.

Answer 1

You have to use item_urls INSIDE the callback in which you first get its value. The only way you can use that value is to use it right there in the callback itself or in a function you call from there and pass it the value.

The issue is that request() is asynchronous and non-blocking. That means that you execute request(), it starts the operation and then it returns and goes right on executing more code (including the code where you try to use item_urls). But, at that point, the callback has not yet been called so it's still undefined.

In the end, the ONLY place that you know item_urls has a value is inside that callback. So, any code that uses that value needs to go inside that callback or get called from inside that callback. That's how asynchronous programming works in node.js.

So, it would need to look like this:

function getitems(){

    request(url, options, (error, res, body) => {
        if (error) {
            return  console.log(error)
        }

        if (!error && res.statusCode == 200) {
            console.log(body);
            // use body.payload.items directly here
            // or call some function here and pass it the value
            // Do not assign it to some higher scoped variable and
            // expect to use it elsewhere.  Can't do that.
        }
    });
}

getitems()

If you want to get the value outside of getitems, then you need to use either a callback, and event or a promise to communicate the asynchronous retrieved value outside of getitems(). You can see a full discussion of why you can't just return the value from getitems() here in this highly popular answer: How do I return the response from an asynchronous call.

---

I don't know what you think doesn't work about the 2nd code block where the code is in a function. I just put this into a file by itself and executed it and it works just fine. So, if you remove item_urls from the equation and you use the result inside the callback like I suggested above, it all works just fine, even in the second code block.

Here's the exact program I just ran:

const request = require('request');

let url = "https://api.warframe.market/v1/items";

let options = {
    json: true
};
function getitems() {
    request(url, options, (error, res, body) => {
        if (error) {
            return console.log(error);
        }

        if (res.statusCode === 200) {
            console.log(body.payload.items);
        }
    });
}
getitems()

This code works. It gives me output like this:

[
  {
    id: '5835a4564b0377e226bdc360',
    url_name: 'axi_c1_intact',
    thumb: 'icons/en/thumbs/Axi_C1_Intact.99413ebefc5b87d57d9e5314265b56de.128x128.png',
    item_name: 'Axi C1 Intact'
  },
  {
    id: '5835a4dd4b0377e226bdc380',
    url_name: 'meso_s3_intact',
    thumb: 'icons/en/thumbs/Meso_S3_Intact.caee59471a7b06ca040f2d257083e008.128x128.png',
    item_name: 'Meso S3 Intact'
  },
  {
    id: '58d8f31c11efe42a5e523215',
    url_name: 'lith_n2_exceptional',
    thumb: 'icons/en/thumbs/Lith_N2_Exceptional.b82a140ba17908be7226fddcecd7bf62.128x128.png',
    item_name: 'Lith N2 Exceptional'
  },
  {
    id: '5936dd6916efa3a742b15f48',
    url_name: 'lith_n3_exceptional',
    thumb: 'icons/en/thumbs/Lith_N3_Exceptional.b82a140ba17908be7226fddcecd7bf62.128x128.png',
    item_name: 'Lith N3 Exceptional'
  },
  {
    id: '5835a4404b0377e226bdc358',
    url_name: 'neo_v4_intact',
    thumb: 'icons/en/thumbs/Neo_V4_Intact.e0e38afae723757487927845b35a81d2.128x128.png',
    item_name: 'Neo V4 Intact'
  },

  .... a whole bunch more items