How to execute parallel query in nodejs?

Question

I have created a function in nodejs which return below object QueryName and Query

{"QueryName":"a","Query":"SELECT something from bar"} 
{"QueryName":"b","Query":"SELECT something from bar"} 
{"QueryName":"c","Query":"SELECT something from bar"} 
{"QueryName":"d","Query":"SELECT something from bar"}

Also, i have created another function to get the query result.

Code is so far

executeQuery(connection, query, config.dbtype, response, function (error, data) {

});

function executeQuery(connection, query, dbtype, response, callback) {
    DB_OPERATIONS.executeOperations(connection, APP_CONSTANT.findall, query, dbtype, null, null, "", function (error, data) {
        callback(error, data);
    });
};

FIND_ALL.findAllRecords(connection, query, dbtype, function(error, result) {
    callback(error, result);
});

Time taking to execute each Query is 5 seconds, total time 20 Seconds.

How can i execute each query parallel in nodejs ?? thenafter instead of 20 seconds, maximum it will takes 5 seconds to get all queries data.

You can fork new processes for running the queries parallel. But keep your machine power in mind. — Rajaprabhu Aravindasamy, Dec 19 '17 at 12:58
every IO operation(like go to db and do query ) in nodejs is parallel by default in node js — Amit Wagner, Dec 19 '17 at 13:08

score 1 · Answer 1 · answered Dec 19 '17 at 15:05

As @zabusa pointed out in the comments you could use promise based functions.

Here are some examples

var queries = [{"QueryName":"a","Query":"SELECT something from bar"} 
{"QueryName":"b","Query":"SELECT something from bar"} 
{"QueryName":"c","Query":"SELECT something from bar"} 
{"QueryName":"d","Query":"SELECT something from bar"}]


//Promisified executeQuery
function executeQueryAsync(connection, query, dbtype, response) {
    return new Promise(function(resolve, reject){
        DB_OPERATIONS.executeOperations(connection, APP_CONSTANT.findall, query.Query, dbtype, null, null, "", function (error, data) {
            var result = {
                QueryName: query.QueryName
            }
            if(error){
                reject(error);
                return;
            }
            result.data = data;
            resolve(result);
        });
    })
};

// Promise with callback
function executeMultipleQueries(callback){

    var promises = [];
    queries.forEach(function(query){
        promises.push(executeQueryAsync(connection, query, config.dbtype, response))
    })

    Promise.all(promises).then(callback)

}

executeMultipleQueries(function(results){
    // results[0].QueryName is a
    // results[1].QueryName is b
    // results[2].QueryName is c
    //actually there is no need to create a result object in the promise, 
    //if we only revolved the data the result array will have them in the order with pushed them in the promises array 
})


//Full Promise

function executeMultipleQueriesPromise(){

    var promises = [];
    queries.forEach(function(query){
        promises.push(executeQueryAsync(connection, query, config.dbtype, response))
    })

    return Promise.all(promises)

}

executeMultipleQueriesPromise().then(function(results){
    // results[0].QueryName is a
    // results[1].QueryName is b
    // results[2].QueryName is c
    //actually there is no need to create a result object in the promise, 
    //if we only revolved the data the result array will have them in the order with pushed them in the promises array 
})

It'll not necessarily run in parallel, promise.all has not this guarantee — v1d3rm3, Mar 09 '22 at 10:42
Hmmm...the code inside the Promise constructor runs when the promise is created and it runs synchronously, so even without ```Promise.All``` everything still runs and its parallel. — Stamos, Mar 09 '22 at 13:35
Concurrecy and Parallelism are different concepts. https://bytearcher.com/articles/parallel-vs-concurrent/ — v1d3rm3, Mar 10 '22 at 13:35
Yea, that debate again. Its true JavaScript is concurrent since its single threaded, but again if you run 10 queries that will take average 5 minutes foreach to complete, do these queries run in the database at the same time? The answer is, yes. — Stamos, Mar 11 '22 at 15:46

score 0 · Answer 2 · answered Dec 19 '17 at 13:00

0

You can use more processes with Cluster. You can see more about multi thread in node in this thread Why is Node.js single threaded?. Hope its help you.

answered Dec 19 '17 at 13:00

Ciro Spaciari

630
5
10

How to execute parallel query in nodejs?

2 Answers2