0

I'm having a bit of trouble with a NodeJS program I have.

So the way my for loop is supposed to function is, it should iterate through the indices from my SQL Database and in each run of the for loop, it's supposed to make a call to another function which makes a GET request.

What it actually does though is it iterates through all of my indices and THEN makes every GET request all at once/very fast, which ends up getting me limited from making anymore API calls. I tried using setTimeout but that doesn't work since it just iterates to the next index and the values end up being undefined.

Also tried a callback but it didn't change anything. I tried looking into async but can't really get my head around what I'm supposed to be doing. Also just a note, nothing is actually being returned. In my other function I'm just updating my SQL DB. I just need the for loop to make an API call per run and only continue after it's complete.

    function callGet() {

    var mysql = require('mysql');
    var con = mysql.createConnection({
        host: "localhost",
        user: "USERNAME",
        password: "PASSWORD",
        database: "sys"
    });

    con.query("SELECT nameAlone FROM testDB", function(err, result, fields) {

       var i;

        for(i = 0; i < result.length; i++){

        var name = result[i].nameAlone;

        var options = {
            method: 'GET',
            url: 'site.com',
            qs: {
                key: 'MYKEY',
                page_size: 10,
                intent: 0,
                itemName: name
            },
            headers: {
                'postman-token': postman-token,
                'cache-control': 'no-cache'
            }
        };  
        getListing(options,result, i);                  //Only being called after the for loop iterates through all of result.length
}
    });
}

An example of what's happening is this, let's say the for loop iterates from 0-2 and each run calls a function that prints out a name. Instead of

index:0, name:Chris, index:1, name:Tony, index:2, name:John

It'll print out:

index:0, index:1, index:2, name:Chris, name:Tony, name:John

Thanks in advance for any help.

Edit: I already attempted all of the answers that applied to my issue in the duplicate question suggested. Trying to mess around with one of the answers there but not too sure if it'll work.

Edit2: All the solutions there didn't work.

Edit3(solution): In case anyone wants a very simple solution,I found this package which was very easy to setup: https://github.com/nathan818fr/async-loop

All I had to do was create an array with my indices and setup the asyncLoop as following:

asyncLoop(indices, function (item, next)
{
    var name = result[item].nameAlone;
    var options = {
        method: 'GET',
        url: 'site.com',
        qs: {
            key: 'MYKEY',
            page_size: 10,
            intent: 0,
            itemName: name
        },
        headers: {
            'postman-token': postman-token,
            'cache-control': 'no-cache'
        }
    };  

    getListing(options,result, item);
    next();
}, function ()
{
    console.log('Finished!');
});
Tony
  • 11
  • 6
  • Possible duplicate of [Javascript async loop processing](https://stackoverflow.com/questions/9772400/javascript-async-loop-processing) – Paul Aug 29 '17 at 15:45
  • Already attempted using setTimeouts with actual delays and the setTimeout 0 trick. – Tony Aug 29 '17 at 15:50
  • @Paul Your proposed question is about how to stop long-running tasks from hogging the thread and starving other tasks. This question appears to be about waiting for async operations to complete for each iteration of a loop. – apsillers Aug 29 '17 at 16:15
  • Exactly what Apsillers said ^^^. There was one solution there that led to some code which could have possibly fixed my issue but it'd didn't do much :/ – Tony Aug 29 '17 at 16:21

1 Answers1

0

You can use my 'ploop' function with promises. ploop basically takes a function that returns a promise, any arguments needed by the function and a function that decides whether ploop is done or not. You do not fill in the 4th argument. Below is an example using ploop to draw random numbers until it finds the number 4:

// Function that returns a promise
var searchForNumber = function(number) {
    return new Promise(function(resolve, reject) {
      setTimeout(function() {
        var min = 1;
        var max = 10;
        var val = Math.floor(Math.random()*(max-min+1)+min);

        console.log('Value is: ' + val.toString());        

        return resolve(val);        
      }, 1000);
    });
};

// fn     : function that should return a promise.
// args   : the arguments that should be passed to fn.
// donefn : function that should check the result of the promise
//    and return true to indicate whether ploop should stop or not.
// promise: A promise value that is used internally by ploop and should never 
//    be passed in by the caller of ploop.
var ploop = function(fn, args, donefn, promise) {
    return (promise || Promise.resolve(true))    
      .then(function() {
          return(fn.apply(null, args));
      })
      .then(function(result) {
        var finished = donefn(result);
        if(finished === true){
           return result;
        } else {
          return ploop(fn, args, donefn, promise);
        }
    });
};

var searchFor = 4;

var donefn = function(result) {
  return result == searchFor;
};

console.log('Searching for: ' + searchFor);
ploop(searchForNumber, [searchFor], donefn)
  .then(function(val) {
    console.log('Finally found! ' + val.toString());  
    process.exit(0);
  })
  .catch(function(err) {
    process.exit(1);
  });

Everything but the ploop function itself is sample code. Here is the demo:

http://jsbin.com/woxetos/edit?js,console

Mike Cheel
  • 12,626
  • 10
  • 72
  • 101
  • I'll give this a try now thanks, have to move around some code to give this a try – Tony Aug 29 '17 at 16:29
  • As I was setting up everything I managed to find a slightly different solution so I'm good now, but you led me to the solution so thank you :) After I finish dealing with my code I'm going to give this solution a try as well. Took me a while because I hadn't dealt with promises before. Thanks again – Tony Aug 29 '17 at 16:53