Mongoose : associate a .project (aggregation) with .find().limit().sort() etc

Question

I have a mongoose Schema looking like this :

var AnswerSchema = new Schema({
  author: {type: Schema.Types.ObjectId, ref: 'User'},
  likes: [{type: Schema.Types.ObjectId, ref: 'User'}],
  date: {type: Date, default: Date.now},
  text: String,
  ....
});

As of now, I query this collection by doing this :

Answer.find({
    author: profileId,
    date: {$lt: fromDate}
  }).sort({date: -1})
    .limit(25)
    .populate('question.sender', 'username username_display name')
    .exec(function (err, answers) {
        *code here*
    });

But I need to add a computed field in this query to know if the visitor already liked the answers of an author (see this question) thanks to the aggregation pipeline. It would look like this :

Answer.aggregate()
    .project({
        "author": 1,            
        "matched": {
            "$eq": [ 
                { 
                    "$size": { 
                        "$ifNull": [
                            { "$setIntersection": [ "$likes", [userId] ] }, 
                            []
                        ] 
                    } 
                },
                1
            ]
        }
    })
    .exec(function (err, docs){
        console.log(docs);
    })

Problem is : I need to run this projection on the documents filtered by the first query only. In other words, I need to combine the find/limit/sort, with the project.

How could I do this ? I've tried chaining the .project() after the first query (before the .exec()), but it doesn't work. Maybe I need to do all the things I do in the first query via the aggregation chain of mongodb ?

EDIT: Here is an example of data-set to try this :

{
    "_id" : ObjectId("56334295e45552c018fc475d"),
    "author" : ObjectId("561f9c319cdd94401ae160ef"),
    "likeCount" : 1,
    "likes" : [ 
        ObjectId("561f9c319cdd94401ae160ef")
    ],
    "text" : "This is the answer content",
    "question" : {
        "questionId" : ObjectId("56334031e45552c018fc4757"),
        "sender" : ObjectId("561f9c319cdd94401ae160ef"),
        "date" : ISODate("2015-10-30T10:02:25.323Z")
    },
    "date" : ISODate("2015-10-30T10:12:37.894Z")
},
{
    "_id" : ObjectId("563246cfa04c1b281b6d97bf"),
    "author" : ObjectId("561f9c319cdd94401ae160ef"),
    "likeCount" : 0,
    "likes" : [],
    "text" : "Bla bla bla",
    "question" : {
        "questionId" : ObjectId("562f9a74a16d6fb418752b37"),
        "sender" : ObjectId("561f9c319cdd94401ae160ef"),
        "date" : ISODate("2015-10-27T15:38:28.337Z")
    },
    "date" : ISODate("2015-10-29T16:18:23.020Z")
}

chridam · Accepted Answer · 2015-11-25T15:53:08.513

5

Use the aggregate fluent API's chaining methods match(), sort() + limit() before the project() method:

Answer.aggregate()
    .match({
        "author": profileId,
        "date": { "$lt": fromDate }
    })
    .sort("-date")
    .limit(25)
    .project({
        "author": 1, "question.sender": 1,          
        "matched": {
            "$eq": [ 
                { 
                    "$size": { 
                        "$ifNull": [
                            { "$setIntersection": [ "$likes", [userId] ] }, 
                            []
                        ] 
                    } 
                },
                1
            ]
        }
    })
    .exec(function (err, docs){
        if (err) throw err;
        console.log(docs);
        var answers = docs.map(function(doc) { return new Answer(doc); });
        Answer.populate(answers, { 
                "path": "question.sender",
                "select": "username username_display name"
            }, function(err, results) {
            if (err) throw err;
            console.log(JSON.stringify(results, undefined, 4 ));
            res.json(results);
        });         
    });

edited Nov 25 '15 at 15:53

answered Nov 25 '15 at 15:13

chridam

100,957
23
236
235

There is still one thing missing : the .populate() doesn't seem to exist. I can't call it when I use the aggregate API. – Hexalyse Nov 25 '15 at 15:23
@Hexalyse Misunderstood the populate requirement in your question. I've updated my answer with an approach that will populate the results from the aggregation. – chridam Nov 25 '15 at 15:54
Thanks ! Now I have a problem... the first query (even before the populate) doesn't return any answer. Which is strange because the query I had before works fine. I tried placing only a match(), and it returns no document, so I think the problem comes from the match(). – Hexalyse Nov 25 '15 at 16:04
A tip I normally use when debugging the aggregation pipeline that is giving unexpected results is to run the aggregation with just the first pipeline operator. If that gives the expected result, add the next. In the example above, you would first try aggregating with just the `.match()` pipeline; if that works, add the `sort()` and so forth. – chridam Nov 25 '15 at 16:08
I edited my comment just before you posted : I tried with only the `.match()` and the query returns no document. So the problem is in the match. Which is strange, because it's the exact same thing that I used in the .find(). EDIT: seems like a casting problem (see http://stackoverflow.com/questions/14551387/cant-use-match-with-mongoose-and-the-aggregation-framework ) – Hexalyse Nov 25 '15 at 16:10
For these types of issues, it would be really helpful if you can edit your question to include a [**Minimal, Complete, and Verifiable example**](http://stackoverflow.com/help/mcve), in other words if you can add some sample documents that we can insert in mongo shell, the expected output and what you are actually getting, that would make it easier to debug. – chridam Nov 25 '15 at 16:14
I've resolved the ObjectId cast issue. But there is still an issue with your solution : the line mapping the returned docs into Answers with "new Answer(doc)" removes the "matched" field, unfortunately. – Hexalyse Nov 26 '15 at 08:39
Thanks @Hexalyse, will update in a bit. Haven't had time to properly test this hence the errors but I appreciate the corrections. – chridam Nov 26 '15 at 08:42
You're welcome. Do you want me to add a data set to my question in order to test and fine the solution to keep the `matched` field while still populating ? – Hexalyse Nov 27 '15 at 09:53
That would be most appreciated, also other SO members can help in given such salient info, thanks! – chridam Nov 27 '15 at 09:57
1

I added a set of two example documents. It should be enough to test my use-case. – Hexalyse Nov 27 '15 at 10:12
I managed to keep the "matched" field just by removing the part of code where you do docs.map() to create mongoose schema objects from simple JSON documents, then going through the `docs` array too add the field to the `results` array with the correct value. – Hexalyse Nov 28 '15 at 13:31

Mongoose : associate a .project (aggregation) with .find().limit().sort() etc

1 Answers1