Get total count along with Mongoose Query skip & limt

Question

I have a json data which contains many objects. I want to limit the data for pagination and I need the total items count. Please help.

Model.find().skip((pageNumber-1)*limit).limit(limit).exec()

I want the count and skipped data in response.

Answer 1

You can use async library for running 2 queries at once. In your case you can run one query to get the number of documents and another for pagination.

Example with 'User' model:

    var async = require('async');
    var User = require('./models/user');

    var countQuery = function(callback){
         User.count({}, function(err, count){
               if(err){ callback(err, null) }
               else{
                   callback(null, count);
                }
         }
    };

    var retrieveQuery = function(callback){
        User.find({}).skip((page-1)*PAGE_LIMIT)
                     .limit(PAGE_LIMIT)
                     .exec(function(err, doc){
                            if(err){ callback(err, null) }
                            else{
                            callback(null, doc);
                            }
                      }
    };

    async.parallel([countQuery, retrieveQuery], function(err, results){
         //err contains the array of error of all the functions
         //results contains an array of all the results
         //results[0] will contain value of doc.length from countQuery function
         //results[1] will contain doc of retrieveQuery function
         //You can send the results as

         res.json({users: results[1], pageLimit: PAGE_LIMIT, page: page, totalCount: results[0]});

    });

async allows you to run a number of queries in parallel depending on the hardware you are using. This would be faster than using 2 independent queries to get count and get the required documents. Hope this helps.

Answer 2

I have solved it with $facet and aggregate the following way in mongoose v3+:

const [{ paginatedResult, [{ totalCount }] }] = await Model.aggregate([{
  $facet: {
    paginatedResult: [
      { $match: query },
      { $skip: skip },
      { $limit: limit }
    ],
    totalCount: [
      { $match: query },
      { $count: 'totalCount' }
    ]
  }
}])

where the totalCount refers the total number of records matching the search query while the paginatedResult is only the paginated slice of them.

Answer 3

The problem with these solutions is that for every request you are doing two queries. This becomes problematic when you have a complex data structure and large data set as performance becomes an issue. Consider instead creating a special function that listens for the /resource?count=true or /resource/count GET methods and returns only the count.

Answer 4

You need to perform 2 queries to achieve that. One to get results and another to get total items amount with .count().

For example code you can watch at on of "paginator" for mongoose mongoose-paginate.

Answer 5

To performe only one query, you may use the find() method associated with promises and array slices. A small example would be:

getPaginated(query, skip, limit){
       return this.model.find(query)
            .lean()
            .then((value)=>{
                if (value.length === 0) return {userMessage: 'Document not found'};
                const count = value.length;

                //skip===0 must be handled
                const start = parseInt(limit)*parseInt(skip - 1);
                const end = start + parseInt(reqQuery.pagesize);

                //slicing the array
                value = value.slice(start,end);

                //could return it another way...
                value.push( { 'querySize': count });
                return value;
              })
              .catch((reason)=>{
                //...handling code
              });
          }