I am trying to build an API that scrapes two different urls at once and sends a JSON response with information from those two urls.
The function below takes a url, makes a request to the url , searches for something and returns the array of that thing
function getData(url) {
let data = [];
request(url, (error,response,body) => {
// Search for foo in the body
data.push(foo);
})
return data;
}
And in the main function, I call the getData function twice with two different urls
app.get('/', (req,res) => {
let x = getData("myUrl/a");
let y = getData("myUrl/b");
let reply = {
"a" : x,
"b" : y
}
res.send(reply);
})
This is basically what I wanna do with my API. But I am getting a blank object! I'm thinking it must be due to the asynchronous thing in JS which I'm not very familiar with.
What should I do to solve this?
EDIT
When I create two different routes and request only a single url, the API works fine