I am using Mongoose with node.js to query our mongoDB. I previously asked a question regarding efficiency (Way to lower memory usage by mongoose when doing query) which helped immensely by teaching me to use lean() and stream().
While memory efficiency has improved greatly, we still have an issue: When grabbing a document like
{
field_a : abc,
field_b : def,
sub_docs : [
{
sub_doc_field_a : xyz,
sub_doc_field_b : afk.
sub_doc_submissions : [
{
submission_id: 0,
score: 1
},
{
submission_id: 1,
score: 1
}...
},...
]
}
I am trying to filter the output by fields (these documents are full of data but the operation we are building uses only some info, from several dozen thousand documents) in order to cut out unnecessary bulk from the data we need to transfer and work with. Specifically, using this example, we use the field filter
{"field_a": 1, "field_b": 1, "sub_docs.sub_doc_submissions.score":1}
That last part is causing the issue, it always returns as an Object, even though lean is returning everything else as raw json. On tracking the memory usage, that part of the returned info takes up way more memory than a simple integer value should. Just to give a visible example of the difference, here's the output from when I run the results through console.log:
{ field_a: abc,
field_b: def,
sub_docs:
[ { sub_doc_submissions: [Object] },
{ sub_doc_submissions: [Object] },
{ sub_doc_submissions: [Object] } ] }
Whereas the ideal output would be like
{ field_a: abc,
field_b: def,
sub_docs:
[ { sub_doc_submissions: {score: 1} },
{ sub_doc_submissions: {score: 1} },
{ sub_doc_submissions: {score: 1} } ] }
or even better
{ field_a: abc,
field_b: def,
sub_docs:
[ { score: 1 },
{ score: 1 },
{ score: 1 } ] }
Since the memory usage comes almost entirely from the query itself, I need to find a way to tell the query to not return the sub_docs.sub_doc_submissions.score values as objects, but rather just their integer values.
Is there a way using mongoose or, if necessary, mongo itself, to do this task such that mongoose will not attempt to instantiate the returned data as Objects despite being in lean mode?