Retrieve paginated results from embedded document in Mongoose

Question

Using mongoose, if I have a Note model, I can retrieve paginated and sorted results using query options on the find function, like so...

   Note.find({ creator: creatorId})
        .select('text')
        .limit(perPage)
        .skip(perPage * page)
        .sort({
            name: 'asc'
        })
        .exec(function(err, notes) {
            Note.count().exec(function(err, count) {
                res.render('notes', {
                    notes: notes,
                    page: page,
                    pages: count / perPage
                })
            })
        });

Can I achieve the same functionality (filter, select, limit, skip, sort etc) if I embed the Note schema within a parent document (notesContainerSchema) like so:

var noteSchema = new Schema({
  creator: { type: String }, 
  text: { type: String }
});

var notesContainerSchema = new Schema({
  key: { type: String, unique: true },
  notes: [ noteSchema ] // note schema is now an array of embedded docs
});

var NotesContainer = db.model('notesContainer', notesContainerSchema);

See [`$slice`](https://docs.mongodb.com/manual/reference/operator/projection/slice/). It has been around from the beginning of MongoDB just for this specific purpose. — Neil Lunn, Jun 17 '17 at 06:19
I understand how `$slice` can be used for the `skip` and `limit`. I don't know how I could also `filter`, `select` and `sort` the data though? — CSharp, Jun 17 '17 at 09:37
If you really **need** all those operations then it might be better to keep as a separate collection. You can alternately use things like [`$lookup`](https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/) as opposed to `.populate()` now to do the "join" on the server when you need the data together. But for embedding, we can do "most" things with an array, re `$filter` etc. However things like `$sort` (dynamically that is )still require `$unwind`. And `$unwind` means performance pain in all scenarios. — Neil Lunn, Jun 17 '17 at 09:45
Embedding has it's place, and is the perfectly valid thing to do in those cases. But if what you do leans more towards items in a separate collection, then that is what you should use. I wrote a general guide to this a while ago [Mongoose populate vs object nesting](https://stackoverflow.com/q/24096546/2313887) — Neil Lunn, Jun 17 '17 at 09:47
Thanks for the info. I'd didn't realise another drawback of embedding was lack of support for sorting when performing pagination. I thought this was such a common requirement there would be operations for it. I'll try the aggregation approach or a separate collection. — CSharp, Jun 17 '17 at 11:02

Bertrand Martel · Accepted Answer · 2017-06-17T10:03:13.337

1

You can use an aggregation with :

a $project stage to $filter notes array with creatorId and $slice the result
a $unwind stage to unwind the notes array
a $sort by names
a $project to select the text field only

In nodeJS, with mongoose :

NotesContainer.aggregate([{
    $project: {
        notes: {
            $slice: [{
                "$filter": {
                    "input": "$notes",
                    "as": "item",
                    "cond": { "$eq": ["$$item.creator", creatorId] }
                }
            }, (perPage * page), perPage]
        }
    }
}, {
    $unwind: "$notes"
}, {
    $sort: {
        "notes.name": 1
    }
}, {
    $project: {
        "text": "$notes.text",
        "_id": 0
    }
}]).exec(function(err, notes) {
    console.log(notes);
});

edited Jun 17 '17 at 10:03

answered Jun 16 '17 at 21:43

Bertrand Martel

42,756
16
135
159

2

Apparently you need to read the [`$slice`](https://docs.mongodb.com/manual/reference/operator/projection/slice/) documentation as well. Please don't suggest complex aggregation statements where simple built in operators do the job. – Neil Lunn Jun 17 '17 at 06:20
thanks for you reply. I get undefined returned when I call `$replaceRoot`. I'm using a database provided by mlab, not sure if it's a version issue. Is there a longer way to get the same result without using `$replaceRoot`? – CSharp Jun 17 '17 at 09:35
1

See the updated post, `$replaceRoot` is in fact not needed and I used `$slice` as suggested by @NeilLunn – Bertrand Martel Jun 17 '17 at 10:08

Retrieve paginated results from embedded document in Mongoose

1 Answers1