10

I am building a page for admin in angular-meteor.

I have published all the records from a collection: "posts" and have taken the subscription of all the records on front end.

$meteor.subscribe('posts');

In the controller, if I select the cursors of all records from minimongo it works fine like:

$scope.posts = $meteor.collection(Posts);

But I want to display pagination, so for that I want limited records at a time like:

$scope.posts = $meteor.collection(function(){

        return Posts.find(

            {}, 
            {
                sort:   {'cDate.timestamp': -1},
                limit:  10
            }
        );
    });

It stucks with the query in minimongo. And the browser hangs.

"posts" collection contains only 500 records. It was working fine when I had 200 records.

Can anyone give me an idea whats wrong with my code and concepts?

EDIT:

Okay! It worked fine when I commented the $sort line from query like this:

$scope.posts = $meteor.collection(function(){

            return Posts.find(

                {}, 
                {
                    //sort:   {'cDate.timestamp': -1},
                    limit:  10
                }
            );
        });

But I need to sort the records. So what should I do now?

EDIT:

Also tried adding index to the sort attribute like this:

db.Posts.ensureIndex({"cDate.timestamp": 1})

Still same issue.

StormTrooper
  • 1,731
  • 4
  • 23
  • 37
  • Did you try adding an index on the cDate.timestamp field. Add index as mentioned in this post, http://stackoverflow.com/questions/9730136/how-to-create-a-nested-index-in-mongodb `db.Posts.ensureIndex({"cDate.timestamp": 1})` – Kishor Feb 04 '16 at 07:24
  • Did try adding an index. Doesn't work. Same issue. Browser is stuck – StormTrooper Feb 04 '16 at 07:37
  • I still think it is an index issue, since it is not having issue if you comment the `sort` line. You can try this suggestion http://stackoverflow.com/questions/28731639/meteor-js-use-hint-to-make-mongo-use-index to make sure the query is using index. If it still doesn't solve the problem then sorry I also don't have idea. – Kishor Feb 04 '16 at 07:42
  • How about publishing only 10 (but sorted) of them? – Radosław Miernik Feb 04 '16 at 08:16
  • @RadosławM So how will I show total records counter if I will have only 10 in minimongo. – StormTrooper Feb 04 '16 at 09:16
  • 1
    Create a method for that. Something like `Meteor.methods({ postsCount: function () { return Posts.find({}, { fields: { _id: 1 } }).count(); } })`. – Radosław Miernik Feb 04 '16 at 09:23
  • The problem reminds me of this issue: https://github.com/meteor/meteor/issues/5633 I'm not familiar with the Angular integration in Meteor, but it could help to wait until the subscription is ready before calling `find`. – klaussner Feb 04 '16 at 09:51
  • @chrisklaussner I don't see a possible solution to my problem in that ticket, yet still I tried to add indexes but no luck :( – StormTrooper Feb 04 '16 at 10:29
  • Sorting performances are horrible client-side as there is no indexing going on whatsoever (except for ID indexing). As such, any sorting should only be done on very small sets of data (such as 5 or 10 posts). When you run the query Meteor first tries to sort everything and only then select the 10 items you are looking for. – Kyll Feb 04 '16 at 11:25
  • @HassanSardar There is a hint in that ticket: "Publish functions send documents one at a time so that queries over the relevant collections need to be re-computed n times when you publish n documents." If you wait until all documents arrive at the client and call `find` afterwards, you won't trigger a repeated and unnecessary sorting of the collection. At least that worked on the reproduction repo provided in the issue. But I agree that you should also consider publishing fewer documents. – klaussner Feb 04 '16 at 12:33
  • @chrisklaussner Well demanding fewer documents can help me out here, but the thing is what will happen when someday I will have to demand large number of documents for sorting and stuff. Issue exists still. – StormTrooper Feb 04 '16 at 12:55
  • It's strange you stuck at 500 records. Me, published > 1500 records and still fine. What kind of record you publish? – asingh Feb 17 '16 at 14:47
  • @yudap You should try to sort your records on client. I published 3000 records with out sorting them on client, works like a charm, but as soon as you sort those record on client from minimongo, than you will see the magic. – StormTrooper Feb 18 '16 at 04:06
  • No. 1500 with sort on client – asingh Feb 18 '16 at 08:37

5 Answers5

3

Change your publication to accept a parameter called pageNumber like this

Meteor.publish('posts', function (pageNumber) {
   var numberOfRecordsPerPage = 10;
   var skipRecords = numberOfRecordsPerPage * (pageNumber - 1);
   return Post.find({
        "user_id": user_id
   }, {
        sort: { 'cDate.timestamp': -1 }
        skip: skipRecords, 
        limit: numberOfRecordsPerPage
   });
});

On client side, I didn't work with angular-meteor much. You can create a pageNumber property under your current scope using this.pageNumber or $scope.pageNumber. Update this pageNumber variable whenever your pagination page is clicked. Whenever this variable is changed, subscribe using the current page number.

If it is using standard blaze template, I would do it using a reactive var or session var in an autorun like this. In template html:

<template name="postsTemplate">
     <ul>
         <!-- you would want to do this list based on total number of records -->
         <li class="pagination" data-value="1">1</li>
         <li class="pagination" data-value="2">2</li>
         <li class="pagination" data-value="3">3</li>
     </ul>
</template>

In template js:

Template.postsTemplate.created = function () {
    var template = this;
    Session.setDefault('paginationPage', 1);
    template.autorun(function () {
       var pageNumber = Session.get('paginationPage');
       Meteor.subscribe('posts', pageNumber);
    });
}

Template.postsTemplate.events(function () {
    'click .pagination': function (ev, template) {
         var target = $(ev.target);
         var pageNumber = target.attr('data-value');
         Session.set('paginationPage', pageNumber);
     }
});

This way, you will have a maximum of 10 records at any point in time on the client, so it will not crash the browser. You might also want to limit the fields that you send to client using something like this

Meteor.publish('posts', function (pageNumber) {
   var numberOfRecordsPerPage = 10;
   var skipRecords = numberOfRecordsPerPage * (pageNumber - 1);
   return Post.find({
        "user_id": user_id
   }, {
        sort: { 'cDate.timestamp': -1 }
        skip: skipRecords, 
        limit: numberOfRecordsPerPage,
        fields: {'message': 1, 'createdBy': 1, 'createdDate': 1 } //The properties inside each document of the posts collection.
   });
});

And finally you will need the total number of records in posts collection on client side, to show the pagination links. You can do it using a different publication and using the observeChanges concept as mentioned in the official documentation here

// server: publish the current size of a collection
Meteor.publish("posts-count", function () {
  var self = this;
  var count = 0;
  var initializing = true;

  // observeChanges only returns after the initial `added` callbacks
  // have run. Until then, we don't want to send a lot of
  // `self.changed()` messages - hence tracking the
  // `initializing` state.
  var handle = Posts.find({}).observeChanges({
    added: function (id) {
      count++;
      if (!initializing)
        self.changed("postsCount", 1, {count: count});
    },
    removed: function (id) {
      count--;
      self.changed("postsCount", 1, {count: count});
    }
    // don't care about changed
  });

  // Instead, we'll send one `self.added()` message right after
  // observeChanges has returned, and mark the subscription as
  // ready.
  initializing = false;
  self.added("postsCount", 1, {count: count});
  self.ready();

  // Stop observing the cursor when client unsubs.
  // Stopping a subscription automatically takes
  // care of sending the client any removed messages.
  self.onStop(function () {
    handle.stop();
  });
});

// client: declare collection to hold count object
PostsCount = new Mongo.Collection("postsCount");

// to get the total number of records and total number of pages
var doc = PostsCount.findOne(); //since we only publish one record with "d == 1", we don't need use query selectors
var count = 0, totalPages = 0;

if (doc) {
    count = doc.count;
    totalPages = Math.ceil(count / 10); //since page number cannot be floating point numbers..
}

Hope this helps.

Kishor
  • 2,659
  • 4
  • 16
  • 34
2

Browser crashing because there is only so much data that it can load in it's cache before itself cashing out. To your question about what happens when you need to demand a large number of documents, take that process away from the client and do as much on the server through optimized publish and subscribe methods / calls. No real reason to load up a ton of documents in the browser cache, as minimongo's convenience is for small data sets and things that don't need to immediately sync (or ever sync) with the server.

  • Ok than tell me how will I show the pagination on client? How will I show the reactive counter of all the records? Or how will I apply the filters which will search from complete database ? – StormTrooper Feb 17 '16 at 06:49
  • It's very possible. First, determine how you're going to subscribe to the posts in the first case. Sounds like you are moving away from minimongo (which is definitely in line with Meteor best practices). – James Mundia Feb 18 '16 at 21:36
  • So if you're going to use a server/client Collection: Are you subscribing at the template level or the router level? What router will you be using? Are you publishing all your documents and then filtering with the subscription? Before you get into pagination these need to be handled first. Some people like to do the code for you here - I wanna see if you can provide us with some snippets of where you are at. If you need help with the above concepts, that's another story, but I think you can do it! – James Mundia Feb 18 '16 at 21:55
2

you should sort on server side, you can find what you are looking for here

meteor publish with limit and sort

Community
  • 1
  • 1
koolaang
  • 397
  • 6
  • 15
2

You need to consider sort and limit strategies:

Sort on the server if you're extracting top values from a large set used by all clients. But usually better first filter by User who needs the data, and sort on the filtered collection. That will reduce the dataset.

Then Publish that sorted / limited subset to the client, and you can do more fine grained / sorting filtering there.

MrE
  • 19,584
  • 12
  • 87
  • 105
2

You should use server side limit instead of client side. That will make your app faster and optimize.

For more please check this link. link here

Pankaj Jatav
  • 2,158
  • 2
  • 14
  • 22