Skip and Limit for pagination for a Mongo aggregate

Question

I am working on pagination in flask(Python framework) using flask-paginate (just for ref)

I am able to achieve pagination for just a find query as below:

from flask_paginate import Pagination
from flask_paginate import get_page_args

def starting_with_letter(letter):
    page, per_page, offset = get_page_args()
    collection_name=letter.lower()+'_collection'
    words=db[collection_name]
    data_db=words.find()
    data=data_db.limit(per_page).skip(offset) '''Here I have achieved the limit and skip'''
    pagination = Pagination(page=page, total=data.count(),per_page=per_page,offset=offset,record_name='words')
    return render_template('startingwords.html',data=data,pagination=pagination)

But I am not able to do the same for the aggregate here:

def test():
    page, per_page, offset = get_page_args()
    cursor_list=[]  '''appending each cursor in iteration of for loop '''
    collections=db.collection_names()
    for collection in collections:
        cursor_objects = db[collection].aggregate([
                {
                    "$match": {
                        "$expr": {"$eq": [{"$strLenCP": "$word"}, 6]}
                    }
                },
                            {"$skip": offset},    
                            {"$limit": per_page}

            ])
        for cursor in cursor_objects:
            cursor_list.append(cursor)
    pagination = Pagination(page=page, total=len(cursor_list),per_page=per_page,offset=offset,record_name='words')
    return render_template('lettersearch.html',data=cursor_list,pagination=pagination)

The results are displayed as :

Here all the 39 results are shown at single page

On hitting page 2 it showed :

Note: By default flask-paginate sets initially per_page as 10 and offset as 0

after referring many links i have tried:

placing skip and limit above match which is wrong any way

Also learnt that limit is always followed by skip

I am stuck with this, Any help is appreciated

Answer 1

Your issue is not with the skip() and limit(); that is working fine. The issue is with your overall logic; you are iterating all 39 collections in the first loop and then appending each result of the aggregation to cursor_list.

I can't figure out the logic of what you are trying to do, as the first example is looking in a words collection and second is looking in all collections for a word field; with that said, you can likely simplify your approach to something like:

offset = 0
per_page = 10
collections = db.list_collection_names()
#
# Add some logic on the collections array to filter what is needed 
#
print(collections[offset:offset+per_page])

EDIT to reflect comments. Full worked example of a function to perform this. No need for an aggregation query - this adds complexity.

from pymongo import MongoClient
from random import randint

db = MongoClient()['testdatabase1']

# Set up some data
for i in range(39):
    coll_name = f'collection{i}'
    db[coll_name].delete_many({}) # Be careful; testing only; this deletes your data
    for k in range (randint(0, 2)):
        db[coll_name].insert_one({'word': '123456'})

# Main function
def test(offset, per_page, word_to_find):
    found = []
    collections = db.list_collection_names()
    for collection in sorted(collections):
        if db[collection].find_one({word_to_find: { '$exists': True}}) is not None:
            found.append(collection)

    print(found[offset:offset+per_page])

test(offset=0, per_page=10, word_to_find='word')