0

I am rewrite the aggregation code for 100k records in my collection "auctions" and i need to create function for the reference of another stack overflow guide

How to increase mongodb BSON document size

but getting error for the

error TS2559: Type '(err: any, auctions: any) => void' has no properties in common with type 'CollectionCreateOptions'

how to get 100k records in BSON limit of more then 16MB in mongoDB

my code

db.collection("auctions",   (err: any, auctions: any): any => {
                    if(err) throw err;
                    const result: any = [];

                    const cursor: any = auctions.aggregate(
                        [
                            {
                                $match: {
                                    status: 'UNPAID'
                                }
                            },
                            {
                                $lookup: {
                                    from: 'bids',
                                    let: { id: '$_id' },
                                    pipeline: [
                                        {
                                            $match: {
                                                $expr: { $and: [{ $eq: ['$auctioncode', '$$id'] }] },
                                            },
                                        },
                                    ],
                                    as: 'bids',
                                },
                            },
                            {
                                $facet: {
                                    products: [{ $skip: 0 }, { $limit: 100000 }],
                                },
                            }
                        ],
                        {
                            "allowDiskUse": true,
                            "cursor": { "batchSize": 20 }
                        }
                    );


                    cursor.on("data",(data: any) => {
                        result.push(data);
                    });

                    cursor.on("end",() => {
                      return result;
                    })
                })
Aravinth E
  • 449
  • 2
  • 6
  • 29

1 Answers1

0

The problem is $facet is pulling all 100k documents into a single array inside of a document.

If you remove the $facet stage, and use the $skip and $limit stages at the end of the pipe, it will return 100k documents via the cursor without exceeding the limit.

Unless, of course, you have a single document in the pipeline that is bigger than 16MB.

Joe
  • 25,000
  • 3
  • 22
  • 44