Async Javascript: Waiting for data to be processed in a for loop before proceeding to a new function

Question

I'm having issues understanding how to work around Javascript's asynchronous behavior in a forEach loop. This issue is quite complex (sorry), but the idea of the loop is as followed:

Loop through every item in an array
Make an HTTP request from a provider script
I then need to multiply every element of the array by a constant
Assign the new array to an item in an object
After the loop, take all the arrays and add them together into one array

The data will be assigned to the indvCoinPortfolioChartData array

I'm looking for any flaws in my event loop. I believe the battle is making this task synchronous, making sure my data is assigned before aggregating data.

The issue

When I'm adding all the arrays together, ONE dataset isn't summed up (I think because it's still being processed after the function is called). There is no error, but it doesn't have all the coin data in the final aggregated array.

This is the issue I see in the aggregatePortfolioChartData function. It begins the for loop with only 2 items in the array, and then later shows 3. The third item was not processed until after the for loop started.

image of console log (logged from aggregatePortfolioChartData function)

debug log when aggregation is successful

var indivCoinPortfolioChartData = {'data': []};
for(var i = 0; i < this.storedCoins.Coins.length; i++)
{  
  let entry = this.storedCoins.Coins[i];
  localThis._data.getChart(entry.symbol, true).subscribe(res => {localThis.generateCoinWatchlistGraph(res, entry);});
  localThis._data.getChart(entry.symbol).subscribe(res => {
    if(entry.holdings > 0)
    {
      let data = res['Data'].map((a) => (a.close * entry.holdings));
      indivCoinPortfolioChartData.data.push({'coinData': data});
      localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
    }
    else
    {
      localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
    }
  });
}

Loaded Coin Data

loadedCoinData(loader, indivCoinPortfolioChartData)
{
  this.coinsWithData++;
  if(this.coinsWithData === this.storedCoins.Coins.length - 1)  
  {
    loader.dismiss();
    this.aggregatePortfolioChartData(indivCoinPortfolioChartData);
  }
}

aggregatePortfolioChartData

aggregatePortfolioChartData(indivCoinPortfolioChartData)
{
  console.log(indivCoinPortfolioChartData);
  var aggregatedPortfolioData = [];
  if(indivCoinPortfolioChartData.data[0].coinData)
  {
    let dataProcessed = 0;
    for(var i = 0; i < indivCoinPortfolioChartData.data[0].coinData.length; i++)
    {
      for(var j = 0; j< indivCoinPortfolioChartData.data.length; j++)
      {
        let data = indivCoinPortfolioChartData.data[j].coinData[i];
        if(data)
        {
          aggregatedPortfolioData[i] = (aggregatedPortfolioData[i] ? aggregatedPortfolioData[i] : 0) + data;
          dataProcessed++;
        }
        else
        {
          dataProcessed++;
        }
      }
      if(dataProcessed === (indivCoinPortfolioChartData.data[0].coinData.length) * (indivCoinPortfolioChartData.data.length))
      {
        console.log(dataProcessed + " data points for portfolio chart");
        this.displayPortfolioChart(aggregatedPortfolioData);
      }
    }
  }
}

Thank you for helping me get through this irksome issue.

Why do `getCoinEquityData` and `setPortfolioChartData` return promises/take callbacks? There's nothing asynchronous going on in there. Or did you post only demo functions? — Bergi, May 06 '18 at 20:09
What is `._data.getChart()`, and how often does it fire the subscribe callback? — Bergi, May 06 '18 at 20:10
In any case, [don't use `forEach`](https://stackoverflow.com/a/37576787/1048572). — Bergi, May 06 '18 at 20:11
@Bergi I'm trying to process data in order so there are no errors. They all require the result returned by the functions. So before setPortfolioChartData can fire, I need getCoinEquityData to finish multiplying the array with a constant. Then after, I use the new array as a parameter for setPortfolioChartData. I can try removing the functions, and have them in the same scope. I thought callbacks were needed because they are dependent variables. ._data.getChart() is the HTTP API request to get an array of points for a chart which is restricted by a time range. In this case, it's 24h — Spuller, May 06 '18 at 20:46
I will try using a for loop or Promise.all and see if I run into similar problems. — Spuller, May 06 '18 at 20:53
No, the callbacks are not needed in `getCoinEquityData` and `setPortfolioChartData`. You should simply `return` the result of your computation. The only function that should return a promise is the asynchronous `getChart`. — Bergi, May 06 '18 at 21:01
@Bergi alright, thanks. I'll give it a go. Someone who is so familiar with synchronous, it can be a bit confusing. I'll get rid of the confusing web of functions and clean and up and let you know if it does the trick. — Spuller, May 06 '18 at 21:15
@Bergi I've made quite a few changes. The problem still persists, but I was able to debug the issue. The for loop in the aggregatePortfolioChartData will run before all the data is assigned to the indivCoinPortfolioChartData object. Any suggestions on how to postpone the function from running until all the data is processed? — Spuller, May 06 '18 at 23:18

Async Javascript: Waiting for data to be processed in a for loop before proceeding to a new function

0 Answers0