Meteor: How to publish cursor that is depending on cursor of other collection

Question

I'm having the following data structure in my Meteor project:
- Users with a set of list-ids that belong to the user (author)
- Lists that actually contain all the data of the list

Now I'm trying to publish all Lists of a user to the client. Here is a simple example:

if (Meteor.isClient) {
    Lists = new Meteor.Collection("lists");

    Deps.autorun(function() {
        Meteor.subscribe("lists");
    });

  Template.hello.greeting = function () {
    return "Test";
  };

  Template.hello.events({
    'click input' : function () {
      if (typeof console !== 'undefined')
        console.log(Lists.find());
    }
  });
}

if (Meteor.isServer) {
    Lists = new Meteor.Collection("lists");

    Meteor.startup(function () {
        if ( Meteor.users.find().count() === 0 ) {
               Accounts.createUser({        //create new user
                   username: 'test',
                   email: 'test@test.com',
                   password: 'test'
               });

               //add list to Lists and id of the list to user
               var user = Meteor.users.findOne({'emails.address' : 'test@test.com', username : 'test'});
               var listid = new Meteor.Collection.ObjectID().valueOf();
               Meteor.users.update(user._id, {$addToSet : {lists : listid}});
               Lists.insert({_id : listid, data : 'content'});
        }
     });


    Meteor.publish("lists", function(){
        var UserListIdsCursor = Meteor.users.find({_id: this.userId}, {limit: 1}).lists;
        if(UserListIdsCursor!=undefined){
            var UserListIds = UserListIdsCursor.fetch();

            return Lists.find({_id : { $in : UserListIds}});
        }
    });

    Meteor.publish("mylists", function(){
        return Meteor.users.find({_id: this.userId}, {limit: 1}).lists;
    });


//at the moment everything is allowed
Lists.allow({
    insert : function(userID)
    {
        return true;
    },
    update : function(userID)
    {
        return true;
    },
    remove : function(userID)
    {
        return true;
    }
});

}

But publishing the Lists doesn't work properly. Any ideas how to fix this? I'm also publishing "mylists" to guarantee that the user has access to the field "lists".

score 3 · Accepted Answer · edited May 23 '17 at 11:52

Solution

Lists = new Meteor.Collection('lists');

if (Meteor.isClient) {
  Tracker.autorun(function() {
    if (Meteor.userId()) {
      Meteor.subscribe('lists');
      Meteor.subscribe('myLists');
    }
  });
}

if (Meteor.isServer) {
  Meteor.startup(function() {
    if (Meteor.users.find().count() === 0) {
      var user = {
        username: 'test',
        email: 'test@test.com',
        password: 'test'
      };

      var userId = Accounts.createUser(user);
      var listId = Lists.insert({data: 'content'});
      Meteor.users.update(userId, {
        $addToSet: {lists: listId}
      });
    }
  });

  Meteor.publish('lists', function() {
    check(this.userId, String);
    var lists = Meteor.users.findOne(this.userId).lists;
    return Lists.find({_id: {$in: lists}});
  });

  Meteor.publish('myLists', function() {
    check(this.userId, String);
    return Meteor.users.find(this.userId, {fields: {lists: 1}});
  });
}

Changes

Declare the Lists collection outside of the client and server (no need to declare it twice).
Ensure the user is logged in when subscribing. (performance enhancement).
When inserting the test user, use the fact that all insert functions return an id (reduces code).
Ensure the user is logged in when publishing.
Simplified lists publish function.
Fixed myLists publish function. A publish needs to return a cursor, an array of cursors, or a falsy value. You can't return an array of ids (which this code doesn't access anyway because you need to do a fetch or a findOne). Important note - this publishes another user document which has the lists field. On the client it will be merged with the existing user document, so only the logged in user will have lists. If you want all users to have the field on the client then I'd recommend just adding it to the user profiles.

Caution: As this is written, if additional list items are appended they will not be published because the lists publish function will only be rerun when the user logs in. To make this work properly, you will need a reactive join.

thank you! reactivity is a important point for me. so i will look at the example you posted and the publish-with-relations package — kerosene, Dec 30 '13 at 12:41
Okay,I tried to publish the data with the publish-with-relations-package (the one that Tom published on GitHub). `Meteor.publish("users", function(){ return Meteor.users.find({_id : this.userId}); });` `Meteor.publish('lists', function(id) { Meteor.publishWithRelations({ handle: this, collection: lists, filter: _id, mappings: [{ key: 'lists', collection: Meteor.users }] }); });` I'm I getting an exception on server:"Exception from sub RKMRo5fCswxinmvB5 ReferenceError: lists is not defined" Although it does exist — kerosene, Dec 30 '13 at 13:43
You need to use `Lists` (your collection instance) not `lists`. If it still doesn't work, I suggest making a new question for this so we can format the code properly. — David Weldon, Dec 31 '13 at 23:38
Thanks, but it didn't work. I opened a new question as you recommended: http://stackoverflow.com/questions/20871496/meteor-error-on-reactive-join-with-publish-with-relations-package — kerosene, Jan 01 '14 at 17:56

score 2 · Answer 2 · edited Jun 20 '20 at 09:12

2

The real problem here is the schema.

Don't store "this user owns these lists" eg, against the users collection. Store "this list is owned by this user"

By changing your example to include an ownerId field on each List then publishing becomes easy - and reactive.

It also removes the need for the myLists publication, as you can just query client side for your lists.

Edit: If your schema also includes a userIds field on each List then publishing is also trivial for non-owners.

Solution

Lists = new Meteor.Collection('lists');

if (Meteor.isClient) {
  Deps.autorun(function() {
    if (Meteor.userId()) {
      Meteor.subscribe('lists.owner');
      Meteor.subscribe('lists.user');
    }
  });
}

if (Meteor.isServer) {
  Lists._ensureIndex('userIds');
  Lists._ensureIndex('ownerId');

  Meteor.startup(function() {
    if (Meteor.users.find().count() === 0) {
      var user = {
        username: 'test',
        email: 'test@test.com',
        password: 'test'
      };

      var userId = Accounts.createUser(user);
      var listId = Lists.insert({data: 'content', ownerId: userId});
    }
  });

  //XX- Oplog tailing in 0.7 doesn't support $ operators - split into two publications -
  //      or change the schema to include the ownerId in the userIds list
  Meteor.publish('lists.user', function() {
    check(this.userId, String);
    return Lists.find({userIds: this.userId});
  });
  Meteor.publish('lists.owner', function() {
    check(this.userId, String);
    return Lists.find({ownerId: this.userId});
  });
}

edited Jun 20 '20 at 09:12

Community

1
1

answered Dec 30 '13 at 01:42

nathan-m

8,875
2
18
29

I chose this schema because I have a field on each list with multiple user ids that are allowed to view this list (and one field called owner). The example above is just a simplification of my code. I chose this schema considering performance. If I would do it the way you proposed I always have to go through all lists to check if a user is allowed to view a list or not. – kerosene Dec 30 '13 at 12:35
It would still be better to store the list of `userIds` with access against the list rather than the user. The publish would be exactly the same as the `lists` publish (as mongodb considers equality as the value existing in an array). Also- this will scale better as users have more and more lists - as the lists collection can be indexed by `ownerId` and `userIds`. – nathan-m Dec 31 '13 at 00:19
I forgot to say that I'm also saving the ids of the users/owner on a list (I didn't put it on the example above). So I'm doing both. Of course this produces redundant data, but I thought it would be the best performant way to realize so scenarios: 1) getting all lists that belong to my user id (get all ids from my profile and get them out of the lists collection) 2) display all users of a list (get all user ids of a list and grab them out of of the user collection). – kerosene Jan 01 '14 at 17:34
I would say scenario (1) is best realised with an index on `Lists` (and is probably just as fast without it for small amounts of data) - rather than duping data. I would also suggest avoiding publishing with relations - you're essentially making extra cursors where you don't need them. (I've added use of `_ensureIndex` and added an `$or` condition to the example) – nathan-m Jan 01 '14 at 23:45
What does small amount of data mean? Imagine there are about ~100 users. A lot of changes are processed in the List collection and everytime a change is made, my-lists-query needs to run again (because of reactivity). – kerosene Jan 02 '14 at 10:06
a small amount of data for indexes - is when it is faster just to sequentially scan a collection, rather than use an index. This could be anywhere from 1000 -> 100000 depending on the complexity of the index/ query etc. Usually the best trick is to `$explain` the query to see if it takes advantage of the index. – nathan-m Jan 03 '14 at 00:04
part 2, is that thanks to oplog tailing in meteor 0.7, it's much better to use simple direct queries than publish with relations. Because when the `users` collection changes, it re-runs the *whole* query on the `Lists` collection. But if you just use a basic query on `Lists`, oplog tailing will just get the new (& update/deleted) results without running any extra queries. = cheap reactivity. (Note: this applies without oplog tailing, as meteor gets notified of changed rows from within the same instance of meteor) – nathan-m Jan 03 '14 at 00:07

score 0 · Answer 3 · answered Dec 29 '13 at 16:12

Meteor.users.find() returns a cursor of many items but you're trying to access .lists with

Meteor.users.find({_id: this.userId}, {limit: 1}).lists;

You need to use findOne instead.

Meteor.users.findOne({_id: this.userId}).lists;

Additionally you're running .fetch() on an array which is stored in the user collection. If this is an array of ._id fields you don't need fetch.

You can't also do .lists in your second publish because its a cursor you have to check lists client side so just use Meteor.users.find(..) on its own since you can only publish cursors.

I used find instead of findOne because I read it guarantees reactivity. Thanks for the second hint — kerosene, Dec 30 '13 at 12:39

Meteor: How to publish cursor that is depending on cursor of other collection

3 Answers3

Solution

Changes

Solution

Linked