Node.JS asynch call back

Question

I am just learning node.js so for my first app like so many people I am creating a music library so I can listen to my music via a web browser from my server at home.

I doing okay... But I am having a hard time understanding how not to create a spagetti nightmare. I am employing promises and using an async library to help with this. but I have this current scenario I am trying to understand how to know when it is finished so it can then preform the necessary call back function.

Basically the gist is I have employed an async library to do two calls one to the db and one to the folder so I can do a comparison of files that I have in the db to the ones indicated on the hard drive. After each async method is done it comes back to the "complete" callback provided by the async library I am using.

At this point I want to run through compare the two list and insert an music files that are missing from the db but where found on the hard drive. then I want to send back to the client the entire list for displaying in a grid.

I am using metamusicdata library to do this. and here is my code so far (untested/not fully complete)

function(err, results) {
    console.log("ASYNC COMPLETED");
    if (err) $this.callback(err, null);
    var db = linq.asEnumerable(results.dbList);
    var files = linq.asEnumerable(results.foldersList);
    var filesToInsert = db.Where(d => !files.Contains(d.LocFile));
    var newMusicList = [];
    for (var file in filesToInsert) {
        if (file.indexOf('mp3') || file.indexOf('mp4')) {
            var fileData = musicMetaData(fs.createReadStream(file), function (err, metadata) {
                if (err) throw err;
                newMusicList.push(new Music ({
                    locFile : file,
                    artist : fileData.artist,
                    album : fileData.album,
                    albumArtist : fileData.albumartist,
                    title : fileData.title,
                    track : fileData.track.no,
                    totalTracks : fileData.track.of,
                    genre : fileData.genre,
                    image : fileData.picture.data,
                    imageType : fileData.picture.format,
                    duration : fileData.duration,
                    createdDate : Date.now(),
                    classification : "unknown"
                }));
                console.log(metadata);
            });
        }
    }
    //insert into database here the array I created.
    var insertMyFiles.
    //query database for the entire list
    var musicListDb;
    //return to the callback so it can send the results back to the client browser.
    return $this.callback(err, musicListDb);
}

The question is how can I set this up to know when the looping through the files is entirely done? So I can take my array created and insert the new entries into the db? So I can then invoke another query and get the update list of the music available?

I can think of what I consider a really messed up way of doing it. Which would be replace the for in with a for i=to length. then in the callback for the musicmetadata at the end check to see if i+1=length if so then invoke the callback inside there.

That does not seem right to me and is rather spagettish in my eyes.

What solution am I failing to see or understand here?

So why did someone down vote my question with out a comment as to how to improve it? Like to know why? — , Nov 06 '16 at 21:27

score 1 · Accepted Answer · answered Nov 06 '16 at 21:03

1

Consider to use async, with it it's really easy to get rid of so-called "callback hell" and much more.

answered Nov 06 '16 at 21:03

aring

3,422
2
22
29

I am already using async as I stated. The area I am working on is the completed function from the async call. And in that instance I do not see any way of using the async library that would be of any use for me with the metadatamusic library call. – Nov 06 '16 at 21:14
1

Your function is too complex, and still you can divide it on multiple separate, logical pieces. Check [this disqussion about async.waterfall](http://stackoverflow.com/questions/25705067/using-async-waterfall-in-node-js) – aring Nov 06 '16 at 21:41
thank you for the additional link that was helpful to get a better understanding... What I did was use the async.eachLimit instead of the for loop. I found if I used the async.Each for some reason it would create blocked io and dramatically slow down response and also winde up with a over flow in the stack. Basically I found i could only process 5 at a time. – Nov 10 '16 at 23:29

score 1 · Answer 2 · answered Nov 07 '16 at 04:45

For that I use promises and/or async/await.

Async/await is simplest. You just have a normal for loop but await on your data.

If you need it to run in parallel you can use something like the promise-all-clear module or just Promise.all with catch()

async function load() {
  const calls = files.map(async f => {
     try { return await meta(f) }
     catch() { return null } 
   });
  let metas = await Promise.all(calls);
  metas = metas.filter(m=>m != null);
  await insert(metas);
}

Node.JS asynch call back

2 Answers2